Index (Google Cloud Dataflow SDK 1.9.1 API)

Google Cloud Dataflow SDK for Java, version 1.9.1

Warning: Cloud Dataflow SDK 1.x for Java is unsupported as of October 16, 2018. See Migrating from Cloud Dataflow SDK 1.x for Java for migration guidance.

Note: If you are using Cloud Dataflow SDK version 2.x, see the Apache Beam API reference documentation.

A B C D E F G H I J K L M N O P R S T U V W X Y 

A

AccumulatingCombineFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Combine.AccumulatingCombineFn

 

accumulatingFiredPanes() - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window

Returns a new Window PTransform that uses the registered WindowFn and Triggering behavior, and that accumulates elements in a pane after they are triggered.

accumulatingFiredPanes() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Bound

Returns a new Window PTransform that uses the registered WindowFn and Triggering behavior, and that accumulates elements in a pane after they are triggered.

accumulatingFiredPanes() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Unbound

Returns a new Window PTransform that uses the registered WindowFn and Triggering behavior, and that accumulates elements in a pane after they are triggered.

add(Iterable<WindowedValue<ElemT>>) - Method in interface com.google.cloud.dataflow.sdk.runners.inprocess.InProcessPipelineRunner.PCollectionViewWriter

 

add(WindowedValue<T>) - Method in interface com.google.cloud.dataflow.sdk.runners.inprocess.InProcessPipelineRunner.UncommittedBundle

Outputs an element to this bundle.

add(Long) - Method in class com.google.cloud.dataflow.sdk.transforms.ApproximateUnique.ApproximateUniqueCombineFn.LargestUnique

Adds a value to the heap, returning whether the value is (large enough to be) in the heap.

add(DisplayData.Item<?>) - Method in interface com.google.cloud.dataflow.sdk.transforms.display.DisplayData.Builder

Register the given display item.

addCollectionToSingletonOutput(String, PValue, PValue) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext

Adds an output with the given name to the previously added CollectionToSingleton Dataflow step, consuming the specified input PValue and producing the specified output PValue.

addComposite(TransformTreeNode) - Method in class com.google.cloud.dataflow.sdk.runners.TransformTreeNode

Adds a composite operation to the transform node.

addElements(T, T...) - Method in class com.google.cloud.dataflow.sdk.testing.TestStream.Builder

Adds the specified elements to the source with timestamp equal to the current watermark.

addElements(TimestampedValue<T>, TimestampedValue<T>...) - Method in class com.google.cloud.dataflow.sdk.testing.TestStream.Builder

Adds the specified elements to the source with the provided timestamps.

addEncodingInput(Coder<?>) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext

Sets the encoding for the current Dataflow step.

addExtensionsFrom(Class<?>...) - Method in class com.google.cloud.dataflow.sdk.coders.Proto2Coder

Deprecated.

use Proto2Coder.withExtensionsFrom(java.lang.Iterable<java.lang.Class<?>>)

addExtensionsFrom(Iterable<Class<?>>) - Method in class com.google.cloud.dataflow.sdk.coders.Proto2Coder

Deprecated.

use Proto2Coder.withExtensionsFrom(java.lang.Iterable<java.lang.Class<?>>)

addIfNotDefault(DisplayData.Item<T>, T) - Method in interface com.google.cloud.dataflow.sdk.transforms.display.DisplayData.Builder

Register the given display item if the value is different than the specified default.

addIfNotNull(DisplayData.Item<?>) - Method in interface com.google.cloud.dataflow.sdk.transforms.display.DisplayData.Builder

Register the given display item if the value is not null.

addInput(String, Boolean) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext

Adds an input with the given name and value to the current Dataflow step.

addInput(String, String) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext

Adds an input with the given name and value to the current Dataflow step.

addInput(String, Long) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext

Adds an input with the given name and value to the current Dataflow step.

addInput(String, PInput) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext

Adds an input with the given name to the previously added Dataflow step, coming from the specified input PValue.

addInput(String, Map<String, Object>) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext

Adds an input that is a dictionary of strings to objects.

addInput(String, List<? extends Map<String, Object>>) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext

Adds an input that is a list of objects.

addInput(TransformTreeNode, PInput) - Method in class com.google.cloud.dataflow.sdk.runners.TransformHierarchy

Adds an input to the given node.

addInput(ApproximateUnique.ApproximateUniqueCombineFn.LargestUnique, T) - Method in class com.google.cloud.dataflow.sdk.transforms.ApproximateUnique.ApproximateUniqueCombineFn

 

addInput(InputT) - Method in interface com.google.cloud.dataflow.sdk.transforms.Combine.AccumulatingCombineFn.Accumulator

Adds the given input value to this accumulator, modifying this accumulator.

addInput(AccumT, InputT) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.AccumulatingCombineFn

 

addInput(double[], Double) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineDoubleFn

 

addInput(Combine.Holder<V>, V) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineFn

 

addInput(int[], Integer) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineIntegerFn

 

addInput(long[], Long) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineLongFn

 

addInput(AccumT, InputT) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.CombineFn

Adds the given input value to the given accumulator, returning the new accumulator value.

addInput(List<V>, V) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.IterableCombineFn

 

addInput(K, AccumT, InputT) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.KeyedCombineFn

Adds the given input value to the given accumulator, returning the new accumulator value.

addInput(Object[], DataT) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedCombineFn

 

addInput(Object[], DataT, CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedCombineFnWithContext

 

addInput(K, Object[], DataT) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedKeyedCombineFn

 

addInput(K, Object[], DataT, CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedKeyedCombineFnWithContext

 

addInput(AccumT, InputT, CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineWithContext.CombineFnWithContext

Adds the given input value to the given accumulator, returning the new accumulator value.

addInput(K, AccumT, InputT, CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineWithContext.KeyedCombineFnWithContext

Adds the given input value to the given accumulator, returning the new accumulator value.

addInput(Top.BoundedHeap<KV<Integer, T>, SerializableComparator<KV<Integer, T>>>, T) - Method in class com.google.cloud.dataflow.sdk.transforms.Sample.FixedSizedSampleFn

 

addInputProducer(PValue, TransformTreeNode) - Method in class com.google.cloud.dataflow.sdk.runners.TransformTreeNode

Adds an input to the transform node.

addInputsRandomly(PerKeyCombineFnRunner<? super K, ? super InputT, AccumT, ?>, K, Iterable<InputT>, Random, DoFn<?, ?>.ProcessContext) - Static method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.TestCombineDoFn

Create a random list of accumulators from the given list of values.

addOutput(String, PValue) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext

Adds an output with the given name to the previously added Dataflow step, producing the specified output PValue, including its Coder if a TypedPValue.

addOutput(InProcessPipelineRunner.UncommittedBundle<?>, InProcessPipelineRunner.UncommittedBundle<?>...) - Method in class com.google.cloud.dataflow.sdk.runners.inprocess.StepTransformResult.Builder

 

addOutput(Collection<InProcessPipelineRunner.UncommittedBundle<?>>) - Method in class com.google.cloud.dataflow.sdk.runners.inprocess.StepTransformResult.Builder

 

addOverrideForClass(Class<?>, DataflowWorkerLoggingOptions.Level) - Method in class com.google.cloud.dataflow.sdk.options.DataflowWorkerLoggingOptions.WorkerLogLevelOverrides

Overrides the default log level for the passed in class.

addOverrideForName(String, DataflowWorkerLoggingOptions.Level) - Method in class com.google.cloud.dataflow.sdk.options.DataflowWorkerLoggingOptions.WorkerLogLevelOverrides

Overrides the default log level for the passed in name.

addOverrideForPackage(Package, DataflowWorkerLoggingOptions.Level) - Method in class com.google.cloud.dataflow.sdk.options.DataflowWorkerLoggingOptions.WorkerLogLevelOverrides

Overrides the default log level for the passed in package.

addStep(PTransform<?, ?>, String) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext

Adds a step to the Dataflow workflow for the given transform, with the given Dataflow step type.

addStep(PTransform<?, ? extends PValue>, Step) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext

Adds a pre-defined step to the Dataflow workflow.

addTraceFor(AbstractGoogleClient, String) - Method in class com.google.cloud.dataflow.sdk.options.GoogleApiDebugOptions.GoogleApiTracer

Creates a GoogleApiDebugOptions.GoogleApiTracer that sets the trace destination on all calls that match the given client type.

addTraceFor(AbstractGoogleClientRequest<?>, String) - Method in class com.google.cloud.dataflow.sdk.options.GoogleApiDebugOptions.GoogleApiTracer

Creates a GoogleApiDebugOptions.GoogleApiTracer that sets the trace traceDestination on all calls that match for the given request type.

addUnprocessedElements(Iterable<? extends WindowedValue<?>>) - Method in class com.google.cloud.dataflow.sdk.runners.inprocess.StepTransformResult.Builder

 

addValue(InputT) - Method in interface com.google.cloud.dataflow.sdk.transforms.Aggregator

Adds a new value into the Aggregator.

addValueInternal(PValue) - Method in class com.google.cloud.dataflow.sdk.Pipeline

Adds the given PValue to this Pipeline.

addValueOnlyOutput(String, PValue) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext

Adds an output with the given name to the previously added Dataflow step, producing the specified output PValue, including its Coder if a TypedPValue.

advance() - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.DatastoreReader

Deprecated.

 

advance() - Method in class com.google.cloud.dataflow.sdk.io.OffsetBasedSource.OffsetBasedReader

 

advance() - Method in class com.google.cloud.dataflow.sdk.io.Source.Reader

Advances the reader to the next valid record.

advance() - Method in class com.google.cloud.dataflow.sdk.io.UnboundedSource.UnboundedReader

Advances the reader to the next valid record.

advanceImpl() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource.FileBasedReader

 

advanceImpl() - Method in class com.google.cloud.dataflow.sdk.io.OffsetBasedSource.OffsetBasedReader

Advances to the next record and returns true, or returns false if there is no next record.

advanceProcessingTime(Duration) - Method in class com.google.cloud.dataflow.sdk.testing.TestStream.Builder

Advance the processing time by the specified amount.

advanceWatermarkTo(Instant) - Method in class com.google.cloud.dataflow.sdk.testing.TestStream.Builder

Advance the watermark of this source to the specified instant.

advanceWatermarkToInfinity() - Method in class com.google.cloud.dataflow.sdk.testing.TestStream.Builder

Advance the watermark to infinity, completing this TestStream.

AfterAll<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing

Create a Trigger that fires and finishes once after all of its sub-triggers have fired.

AfterDelayFromFirstElement<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing

A base class for triggers that happen after a processing time delay from the arrival of the first element in a pane.

AfterDelayFromFirstElement(TimeDomain, List<SerializableFunction<Instant, Instant>>) - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.AfterDelayFromFirstElement

 

AfterEach<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing

A composite Trigger that executes its sub-triggers in order.

afterElement(WindowedValue<T>) - Method in interface com.google.cloud.dataflow.sdk.runners.inprocess.ModelEnforcement

Called after a call to TransformEvaluator.processElement(WindowedValue) on the provided WindowedValue.

afterFinish(InProcessPipelineRunner.CommittedBundle<T>, InProcessTransformResult, Iterable<? extends InProcessPipelineRunner.CommittedBundle<?>>) - Method in interface com.google.cloud.dataflow.sdk.runners.inprocess.ModelEnforcement

Called after a bundle has been completed and TransformEvaluator.finishBundle() has been called, producing the provided InProcessTransformResult and output bundles.

AfterFirst<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing

Create a composite Trigger that fires once after at least one of its sub-triggers have fired.

AfterPane<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing

Triggers that fire based on properties of the elements in the current pane.

AfterProcessingTime<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing

AfterProcessingTime triggers fire based on the current processing time.

AfterWatermark<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing

AfterWatermark triggers fire based on progress of the system watermark.

AfterWatermark.AfterWatermarkEarly<W extends BoundedWindow> - Interface in com.google.cloud.dataflow.sdk.transforms.windowing

Interface for building an AfterWatermarkTrigger with early firings already filled in.

AfterWatermark.AfterWatermarkLate<W extends BoundedWindow> - Interface in com.google.cloud.dataflow.sdk.transforms.windowing

Interface for building an AfterWatermarkTrigger with late firings already filled in.

AfterWatermark.FromEndOfWindow<W extends BoundedWindow> - Class in com.google.cloud.dataflow.sdk.transforms.windowing

A watermark trigger targeted relative to the end of the window.

Aggregator<InputT,OutputT> - Interface in com.google.cloud.dataflow.sdk.transforms

An Aggregator<InputT> enables monitoring of values of type InputT, to be combined across all bundles.

AggregatorPipelineExtractor - Class in com.google.cloud.dataflow.sdk.runners

Retrieves Aggregators at each ParDo and returns a Map of Aggregator to the PTransforms in which it is present.

AggregatorPipelineExtractor(Pipeline) - Constructor for class com.google.cloud.dataflow.sdk.runners.AggregatorPipelineExtractor

Creates an AggregatorPipelineExtractor for the given Pipeline.

AggregatorRetrievalException - Exception in com.google.cloud.dataflow.sdk.runners

Signals that an exception has occurred while retrieving Aggregators.

AggregatorRetrievalException(String, Throwable) - Constructor for exception com.google.cloud.dataflow.sdk.runners.AggregatorRetrievalException

Constructs a new AggregatorRetrievalException with the specified detail message and cause.

AggregatorRetriever - Class in com.google.cloud.dataflow.sdk.transforms

An internal class for extracting Aggregators from DoFns.

AggregatorValues<T> - Class in com.google.cloud.dataflow.sdk.runners

A collection of values associated with an Aggregator.

AggregatorValues() - Constructor for class com.google.cloud.dataflow.sdk.runners.AggregatorValues

 

alignedTo(Duration, Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterDelayFromFirstElement

Aligns timestamps to the smallest multiple of size since the offset greater than the timestamp.

alignedTo(Duration) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterDelayFromFirstElement

Aligns the time to be the smallest multiple of size greater than the timestamp since the epoch.

ALL_CONTEXTS - Static variable in class com.google.cloud.dataflow.sdk.testing.CoderProperties

All the contexts, for use in test cases.

ALL_KEYS - Static variable in class com.google.cloud.dataflow.sdk.io.range.ByteKeyRange

The range of all keys, with empty start and end keys.

allowsDynamicSplitting() - Method in class com.google.cloud.dataflow.sdk.io.OffsetBasedSource

Whether this source should allow dynamic splitting of the offset ranges.

and(TupleTag<V>, List<V>) - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult

Returns a new CoGbkResult based on this, with the given tag and given data added to it.

and(TupleTag<V>, PCollection<KV<K, V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.join.KeyedPCollectionTuple

Returns a new KeyedPCollectionTuple<K> that is the same as this, appended with the given PCollection.

and(PCollection.IsBounded) - Method in enum com.google.cloud.dataflow.sdk.values.PCollection.IsBounded

Returns the composed IsBounded property.

and(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.values.PCollectionList

Returns a new PCollectionList that has all the PCollections of this PCollectionList plus the given PCollection appended to the end.

and(Iterable<PCollection<T>>) - Method in class com.google.cloud.dataflow.sdk.values.PCollectionList

Returns a new PCollectionList that has all the PCollections of this PCollectionList plus the given PCollections appended to the end, in order.

and(TupleTag<T>, PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.values.PCollectionTuple

Returns a new PCollectionTuple that has each PCollection and TupleTag of this PCollectionTuple plus the given PCollection associated with the given TupleTag.

and(TupleTag<?>) - Method in class com.google.cloud.dataflow.sdk.values.TupleTagList

Returns a new TupleTagList that has all the TupleTags of this TupleTagList plus the given TupleTag appended to the end.

and(List<TupleTag<?>>) - Method in class com.google.cloud.dataflow.sdk.values.TupleTagList

Returns a new TupleTagList that has all the TupleTags of this TupleTagList plus the given TupleTags appended to the end, in order.

any(long) - Static method in class com.google.cloud.dataflow.sdk.transforms.Sample

Sample#any(long) takes a PCollection<T> and a limit, and produces a new PCollection<T> containing up to limit elements of the input PCollection.

applicableTo(PCollection<?>) - Static method in class com.google.cloud.dataflow.sdk.transforms.GroupByKey

 

ApplicationNameOptions - Interface in com.google.cloud.dataflow.sdk.options

Options that allow setting the application name.

AppliedPTransform<InputT extends PInput,OutputT extends POutput,TransformT extends PTransform<? super InputT,OutputT>> - Class in com.google.cloud.dataflow.sdk.transforms

Represents the application of a PTransform to a specific input to produce a specific output.

apply(InputT) - Method in interface com.google.cloud.dataflow.sdk.coders.DelegateCoder.CodingFunction

 

apply(PInput) - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Read.Bound

 

apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.io.AvroIO.Write.Bound

 

apply(PInput) - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Read.Bound

 

apply(PCollection<TableRow>) - Method in class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.Bound

 

apply(PBegin) - Method in class com.google.cloud.dataflow.sdk.io.bigtable.BigtableIO.Read

 

apply(PCollection<KV<ByteString, Iterable<Mutation>>>) - Method in class com.google.cloud.dataflow.sdk.io.bigtable.BigtableIO.Write

 

apply(PBegin) - Method in class com.google.cloud.dataflow.sdk.io.CountingInput.BoundedCountingInput

 

apply(PBegin) - Method in class com.google.cloud.dataflow.sdk.io.CountingInput.UnboundedCountingInput

 

apply(PBegin) - Method in class com.google.cloud.dataflow.sdk.io.datastore.DatastoreV1.Read

Applies this PTransform on the given InputT, and returns its Output.

apply(PInput) - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Read.Bound

 

apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.Write.Bound

 

apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.io.PubsubUnboundedSink

 

apply(PBegin) - Method in class com.google.cloud.dataflow.sdk.io.PubsubUnboundedSource

 

apply(PInput) - Method in class com.google.cloud.dataflow.sdk.io.Read.Bounded

 

apply(PInput) - Method in class com.google.cloud.dataflow.sdk.io.Read.Unbounded

 

apply(PInput) - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Read.Bound

 

apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.io.TextIO.Write.Bound

 

apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.io.Write.Bound

 

apply(PTransform<? super PBegin, OutputT>) - Method in class com.google.cloud.dataflow.sdk.Pipeline

Like Pipeline.apply(String, PTransform) but the transform node in the Pipeline graph will be named according to PTransform.getName().

apply(String, PTransform<? super PBegin, OutputT>) - Method in class com.google.cloud.dataflow.sdk.Pipeline

Adds a root PTransform, such as Read or Create, to this Pipeline.

apply(PTransform<InputT, OutputT>, InputT) - Method in class com.google.cloud.dataflow.sdk.runners.BlockingDataflowPipelineRunner

 

apply(PTransform<InputT, OutputT>, InputT) - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner

Applies the given transform to the input.

apply(PTransform<InputT, OutputT>, InputT) - Method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner

 

apply(InputT) - Method in class com.google.cloud.dataflow.sdk.runners.inprocess.ForwardingPTransform

 

apply(PTransform<InputT, OutputT>, InputT) - Method in class com.google.cloud.dataflow.sdk.runners.inprocess.InProcessPipelineRunner

 

apply(PTransform<InputT, OutputT>, InputT) - Method in class com.google.cloud.dataflow.sdk.runners.PipelineRunner

Applies a transform to the given input, returning the output.

apply(PTransform<InputT, OutputT>, InputT) - Method in class com.google.cloud.dataflow.sdk.runners.TemplatingDataflowPipelineRunner

 

apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert.GroupThenAssert

 

apply(PCollection<Iterable<T>>) - Method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert.GroupThenAssertForSingleton

 

apply(PBegin) - Method in class com.google.cloud.dataflow.sdk.testing.DataflowAssert.OneSideInputAssert

 

apply(PTransform<InputT, OutputT>, InputT) - Method in class com.google.cloud.dataflow.sdk.testing.TestDataflowPipelineRunner

 

apply(PBegin) - Method in class com.google.cloud.dataflow.sdk.testing.TestStream

 

apply(double, double) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineDoubleFn

Applies the binary operation to the two operands, returning the result.

apply(V, V) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineFn

Applies the binary operation to the two operands, returning the result.

apply(int, int) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineIntegerFn

Applies the binary operation to the two operands, returning the result.

apply(long, long) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineLongFn

Applies the binary operation to the two operands, returning the result.

apply(Iterable<? extends InputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.CombineFn

Applies this CombineFn to a collection of input values to produce a combined output value.

apply(PCollection<InputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.Globally

 

apply(PCollection<InputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.GloballyAsSingletonView

 

apply(PCollection<? extends KV<K, ? extends Iterable<InputT>>>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.GroupedValues

 

apply(K, Iterable<? extends InputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.KeyedCombineFn

Applies this KeyedCombineFn to a key and a collection of input values to produce a combined output value.

apply(PCollection<KV<K, InputT>>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.PerKey

 

apply(PCollection<KV<K, InputT>>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.PerKeyWithHotKeyFanout

 

apply(K, Iterable<? extends InputT>, CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineWithContext.KeyedCombineFnWithContext

Applies this KeyedCombineFnWithContext to a key and a collection of input values to produce a combined output value.

apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.Count.PerElement

 

apply(PInput) - Method in class com.google.cloud.dataflow.sdk.transforms.Create.TimestampedValues

 

apply(PInput) - Method in class com.google.cloud.dataflow.sdk.transforms.Create.Values

 

apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.Filter

 

apply(PCollection<InputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.FlatMapElements

 

apply(PCollection<? extends Iterable<T>>) - Method in class com.google.cloud.dataflow.sdk.transforms.Flatten.FlattenIterables

 

apply(PCollectionList<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.Flatten.FlattenPCollectionList

 

apply(PCollection<KV<K, V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.GroupByKey

 

apply(PCollection<KV<K, Iterable<WindowedValue<V>>>>) - Method in class com.google.cloud.dataflow.sdk.transforms.GroupByKey.GroupAlsoByWindow

 

apply(PCollection<KV<K, V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.GroupByKey.GroupByKeyOnly

 

apply(PCollection<KV<K, V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.GroupByKey.ReifyTimestampsAndWindows

 

apply(PCollection<KV<K, Iterable<WindowedValue<V>>>>) - Method in class com.google.cloud.dataflow.sdk.transforms.GroupByKey.SortValuesByTimestamp

 

apply(PCollection<? extends InputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.IntraBundleParallelization.Bound

 

apply(KeyedPCollectionTuple<K>) - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGroupByKey

 

apply(PTransform<KeyedPCollectionTuple<K>, OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.join.KeyedPCollectionTuple

Like KeyedPCollectionTuple.apply(String, PTransform) but defaulting to the name provided by the PTransform.

apply(String, PTransform<KeyedPCollectionTuple<K>, OutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.join.KeyedPCollectionTuple

Applies the given PTransform to this input KeyedPCollectionTuple and returns its OutputT.

apply(PCollection<? extends KV<K, ?>>) - Method in class com.google.cloud.dataflow.sdk.transforms.Keys

 

apply(PCollection<KV<K, V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.KvSwap

 

apply(PCollection<InputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.MapElements

 

apply(T, T) - Method in class com.google.cloud.dataflow.sdk.transforms.Max.MaxFn

 

apply(T, T) - Method in class com.google.cloud.dataflow.sdk.transforms.Min.MinFn

 

apply(PCollection<? extends InputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.Bound

 

apply(PCollection<? extends InputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.ParDo.BoundMulti

 

apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.Partition

 

apply(InputT) - Method in class com.google.cloud.dataflow.sdk.transforms.PTransform

Applies this PTransform on the given InputT, and returns its Output.

apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.RemoveDuplicates

 

apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.RemoveDuplicates.WithRepresentativeValues

 

apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.Sample.SampleAny

 

apply(InputT) - Method in interface com.google.cloud.dataflow.sdk.transforms.SerializableFunction

Returns the result of invoking this function on the given input.

apply(double, double) - Method in class com.google.cloud.dataflow.sdk.transforms.Sum.SumDoubleFn

 

apply(int, int) - Method in class com.google.cloud.dataflow.sdk.transforms.Sum.SumIntegerFn

 

apply(long, long) - Method in class com.google.cloud.dataflow.sdk.transforms.Sum.SumLongFn

 

apply(PCollection<? extends KV<?, V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.Values

 

apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.View.AsIterable

 

apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.View.AsList

 

apply(PCollection<KV<K, V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.View.AsMap

 

apply(PCollection<KV<K, V>>) - Method in class com.google.cloud.dataflow.sdk.transforms.View.AsMultimap

 

apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.View.AsSingleton

 

apply(PCollection<ElemT>) - Method in class com.google.cloud.dataflow.sdk.transforms.View.CreatePCollectionView

 

apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Bound

 

apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Window.Remerge

 

apply(PCollection<V>) - Method in class com.google.cloud.dataflow.sdk.transforms.WithKeys

 

apply(PCollection<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.WithTimestamps

 

apply(PTransform<? super PBegin, OutputT>) - Method in class com.google.cloud.dataflow.sdk.values.PBegin

Like PBegin.apply(String, PTransform) but defaulting to the name of the PTransform.

apply(String, PTransform<? super PBegin, OutputT>) - Method in class com.google.cloud.dataflow.sdk.values.PBegin

Applies the given PTransform to this input PBegin, using name to identify this specific application of the transform.

apply(PTransform<? super PCollection<T>, OutputT>) - Method in class com.google.cloud.dataflow.sdk.values.PCollection

Like PCollection.apply(String, PTransform) but defaulting to the name of the PTransform.

apply(String, PTransform<? super PCollection<T>, OutputT>) - Method in class com.google.cloud.dataflow.sdk.values.PCollection

Applies the given PTransform to this input PCollection, using name to identify this specific application of the transform.

apply(PTransform<PCollectionList<T>, OutputT>) - Method in class com.google.cloud.dataflow.sdk.values.PCollectionList

Like PCollectionList.apply(String, PTransform) but defaulting to the name of the PTransform.

apply(String, PTransform<PCollectionList<T>, OutputT>) - Method in class com.google.cloud.dataflow.sdk.values.PCollectionList

Applies the given PTransform to this input PCollectionList, using name to identify this specific application of the transform.

apply(PTransform<PCollectionTuple, OutputT>) - Method in class com.google.cloud.dataflow.sdk.values.PCollectionTuple

Like PCollectionTuple.apply(String, PTransform) but defaulting to the name of the PTransform.

apply(String, PTransform<PCollectionTuple, OutputT>) - Method in class com.google.cloud.dataflow.sdk.values.PCollectionTuple

Applies the given PTransform to this input PCollectionTuple, using name to identify this specific application of the transform.

applyTransform(InputT, PTransform<? super InputT, OutputT>) - Static method in class com.google.cloud.dataflow.sdk.Pipeline

Like Pipeline.applyTransform(String, PInput, PTransform) but defaulting to the name provided by the PTransform.

applyTransform(String, InputT, PTransform<? super InputT, OutputT>) - Static method in class com.google.cloud.dataflow.sdk.Pipeline

Applies the given PTransform to this input InputT and returns its OutputT.

ApproximateQuantiles - Class in com.google.cloud.dataflow.sdk.transforms

PTransforms for getting an idea of a PCollection's data distribution using approximate N-tiles (e.g.

ApproximateQuantiles.ApproximateQuantilesCombineFn<T,ComparatorT extends Comparator<T> & Serializable> - Class in com.google.cloud.dataflow.sdk.transforms

The ApproximateQuantilesCombineFn combiner gives an idea of the distribution of a collection of values using approximate N-tiles.

ApproximateUnique - Class in com.google.cloud.dataflow.sdk.transforms

PTransforms for estimating the number of distinct elements in a PCollection, or the number of distinct values associated with each key in a PCollection of KVs.

ApproximateUnique() - Constructor for class com.google.cloud.dataflow.sdk.transforms.ApproximateUnique

 

ApproximateUnique.ApproximateUniqueCombineFn<T> - Class in com.google.cloud.dataflow.sdk.transforms

CombineFn that computes an estimate of the number of distinct values that were combined.

ApproximateUnique.ApproximateUniqueCombineFn.LargestUnique - Class in com.google.cloud.dataflow.sdk.transforms

A heap utility class to efficiently track the largest added elements.

ApproximateUniqueCombineFn(long, Coder<T>) - Constructor for class com.google.cloud.dataflow.sdk.transforms.ApproximateUnique.ApproximateUniqueCombineFn

 

areAllSubtriggersFinished() - Method in interface com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerInfo

Returns true if all the sub-triggers of the current trigger are marked finished.

as(Class<T>) - Method in interface com.google.cloud.dataflow.sdk.options.PipelineOptions

Transforms this object into an object of type <T> saving each property that has been manipulated.

as(Class<T>) - Static method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory

Creates and returns an object that implements <T>.

as(Class<T>) - Method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory.Builder

Creates and returns an object that implements <T> using the values configured on this builder during construction.

asCloudObject() - Method in class com.google.cloud.dataflow.sdk.coders.AvroCoder

 

asCloudObject() - Method in interface com.google.cloud.dataflow.sdk.coders.Coder

Returns the CloudObject that represents this Coder.

asCloudObject() - Method in class com.google.cloud.dataflow.sdk.coders.CustomCoder

Returns the CloudObject that represents this Coder.

asCloudObject() - Method in class com.google.cloud.dataflow.sdk.coders.IterableCoder

 

asCloudObject() - Method in class com.google.cloud.dataflow.sdk.coders.JAXBCoder

 

asCloudObject() - Method in class com.google.cloud.dataflow.sdk.coders.KvCoder

 

asCloudObject() - Method in class com.google.cloud.dataflow.sdk.coders.Proto2Coder

Deprecated.

 

asCloudObject() - Method in class com.google.cloud.dataflow.sdk.coders.protobuf.ProtoCoder

 

asCloudObject() - Method in class com.google.cloud.dataflow.sdk.coders.SerializableCoder

 

asCloudObject() - Method in class com.google.cloud.dataflow.sdk.coders.StandardCoder

 

asCloudObject() - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult.CoGbkResultCoder

 

asCloudObject() - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResultSchema

 

asCloudObject() - Method in class com.google.cloud.dataflow.sdk.values.TupleTag

 

asIterable() - Static method in class com.google.cloud.dataflow.sdk.transforms.View

Returns a View.AsIterable transform that takes a PCollection as input and produces a PCollectionView mapping each window to an Iterable of the values in that window.

asKeyedFn() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.CombineFn

 

asKeyedFn() - Method in interface com.google.cloud.dataflow.sdk.transforms.CombineFnBase.GlobalCombineFn

Converts this GloballyCombineFn into an equivalent CombineFnBase.PerKeyCombineFn that ignores the keys passed to it and combines the values according to this GloballyCombineFn.

asKeyedFn() - Method in class com.google.cloud.dataflow.sdk.transforms.CombineWithContext.CombineFnWithContext

 

asList() - Static method in class com.google.cloud.dataflow.sdk.transforms.View

Returns a View.AsList transform that takes a PCollection and returns a PCollectionView mapping each window to a List containing all of the elements in the window.

asMap() - Method in class com.google.cloud.dataflow.sdk.transforms.display.DisplayData

 

asMap() - Static method in class com.google.cloud.dataflow.sdk.transforms.View

Returns a View.AsMap transform that takes a PCollection<KV<K, V>> as input and produces a PCollectionView mapping each window to a Map<K, V>.

asMultimap() - Static method in class com.google.cloud.dataflow.sdk.transforms.View

Returns a View.AsMultimap transform that takes a PCollection<KV<K, V>> as input and produces a PCollectionView mapping each window to its contents as a Map<K, Iterable<V>> for use as a side input.

asOutputReference(PValue) - Method in interface com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator.TranslationContext

Encode a PValue reference as an output reference.

asPath() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.PubsubSubscription

Returns the string representation of this subscription as a path used in the Cloud Pub/Sub API.

asPath() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.PubsubTopic

Returns the string representation of this topic as a path used in the Cloud Pub/Sub API.

assertSourcesEqualReferenceSource(BoundedSource<T>, List<? extends BoundedSource<T>>, PipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.testing.SourceTestUtils

Given a reference Source and a list of Sources, assert that the union of the records read from the list of sources is equal to the records read from the reference source.

assertSplitAtFractionBehavior(BoundedSource<T>, int, double, SourceTestUtils.ExpectedSplitOutcome, PipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.testing.SourceTestUtils

Asserts that the source's reader either fails to splitAtFraction(fraction) after reading numItemsToReadBeforeSplit items, or succeeds in a way that is consistent according to SourceTestUtils.assertSplitAtFractionSucceedsAndConsistent(com.google.cloud.dataflow.sdk.io.BoundedSource<T>, int, double, com.google.cloud.dataflow.sdk.options.PipelineOptions).

assertSplitAtFractionExhaustive(BoundedSource<T>, PipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.testing.SourceTestUtils

Asserts that for each possible start position, BoundedSource.BoundedReader#splitAtFraction at every interesting fraction (halfway between two fractions that differ by at least one item) can be called successfully and the results are consistent if a split succeeds.

assertSplitAtFractionFails(BoundedSource<T>, int, double, PipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.testing.SourceTestUtils

Asserts that the source's reader fails to splitAtFraction(fraction) after reading numItemsToReadBeforeSplit items.

assertSplitAtFractionSucceedsAndConsistent(BoundedSource<T>, int, double, PipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.testing.SourceTestUtils

Verifies some consistency properties of BoundedSource.BoundedReader#splitAtFraction on the given source.

assertUnstartedReaderReadsSameAsItsSource(BoundedSource.BoundedReader<T>, PipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.testing.SourceTestUtils

Assert that a Reader returns a Source that, when read from, produces the same records as the reader.

AssignContext() - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn.AssignContext

 

assignedWindows(WindowFn<T, W>, long) - Static method in class com.google.cloud.dataflow.sdk.testing.WindowFnTestUtils

 

assignOutputTime(Instant, W) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFn

Returns the output timestamp to use for data depending on the given inputTimestamp in the specified window.

assignOutputTime(W) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFn.DependsOnlyOnWindow

Returns the output timestamp to use for data in the specified window.

assignOutputTime(Instant, W) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFn.DependsOnlyOnWindow

Returns the output timestamp to use for data depending on the given inputTimestamp in the specified window.

assignsToSingleWindow() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.GlobalWindows

 

assignsToSingleWindow() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.PartitioningWindowFn

 

assignsToSingleWindow() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn

Returns true if this WindowFn assigns each element to a single window.

assignWindow(Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.DaysWindows

 

assignWindow(Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.MonthsWindows

 

assignWindow(Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.YearsWindows

 

assignWindow(Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.FixedWindows

 

assignWindow(Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.PartitioningWindowFn

Returns the single window to which elements with this timestamp belong.

assignWindows(WindowFn<Object, GlobalWindow>.AssignContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.GlobalWindows

 

assignWindows(WindowFn<Object, W>.AssignContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.InvalidWindows

 

assignWindows(WindowFn<T, W>.AssignContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.PartitioningWindowFn

 

assignWindows(WindowFn<Object, IntervalWindow>.AssignContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Sessions

 

assignWindows(WindowFn<Object, IntervalWindow>.AssignContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.SlidingWindows

 

assignWindows(WindowFn<T, W>.AssignContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn

Given a timestamp and element, returns the set of windows into which it should be placed.

asSingleton() - Static method in class com.google.cloud.dataflow.sdk.transforms.View

Returns a View.AsSingleton transform that takes a PCollection with a single value per window as input and produces a PCollectionView that returns the value in the main input window when read as a side input.

asSingletonView() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.Globally

Returns a PTransform that produces a PCollectionView whose elements are the result of combining elements per-window in the input PCollection.

asV1Beta1Path() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.PubsubSubscription

Deprecated.

the v1beta1 API for Cloud Pub/Sub is deprecated.

asV1Beta1Path() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.PubsubTopic

Deprecated.

the v1beta1 API for Cloud Pub/Sub is deprecated.

asV1Beta2Path() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.PubsubSubscription

Deprecated.

the v1beta2 API for Cloud Pub/Sub is deprecated.

asV1Beta2Path() - Method in class com.google.cloud.dataflow.sdk.io.PubsubIO.PubsubTopic

Deprecated.

the v1beta2 API for Cloud Pub/Sub is deprecated.

AtomicCoder<T> - Class in com.google.cloud.dataflow.sdk.coders

A Coder that has no component Coders or other state.

AtomicCoder() - Constructor for class com.google.cloud.dataflow.sdk.coders.AtomicCoder

 

AtomicLongFactory() - Constructor for class com.google.cloud.dataflow.sdk.options.PipelineOptions.AtomicLongFactory

 

AvroCoder<T> - Class in com.google.cloud.dataflow.sdk.coders

A Coder using Avro binary format.

AvroCoder(Class<T>, Schema) - Constructor for class com.google.cloud.dataflow.sdk.coders.AvroCoder

 

AvroIO - Class in com.google.cloud.dataflow.sdk.io

PTransforms for reading and writing Avro files.

AvroIO.Read - Class in com.google.cloud.dataflow.sdk.io

A root PTransform that reads from an Avro file (or multiple Avro files matching a pattern) and returns a PCollection containing the decoding of each record.

AvroIO.Read.Bound<T> - Class in com.google.cloud.dataflow.sdk.io

A PTransform that reads from an Avro file (or multiple Avro files matching a pattern) and returns a bounded PCollection containing the decoding of each record.

AvroIO.Write - Class in com.google.cloud.dataflow.sdk.io

A root PTransform that writes a PCollection to an Avro file (or multiple Avro files matching a sharding pattern).

AvroIO.Write.Bound<T> - Class in com.google.cloud.dataflow.sdk.io

A PTransform that writes a bounded PCollection to an Avro file (or multiple Avro files matching a sharding pattern).

AvroReader(AvroSource<T>) - Constructor for class com.google.cloud.dataflow.sdk.io.AvroSource.AvroReader

Reads Avro records of type T from the specified source.

AvroSource<T> - Class in com.google.cloud.dataflow.sdk.io

A FileBasedSource for reading Avro files.

AvroSource.AvroReader<T> - Class in com.google.cloud.dataflow.sdk.io

A BlockBasedSource.BlockBasedReader for reading blocks from Avro files.

awaitCompletion() - Method in class com.google.cloud.dataflow.sdk.runners.inprocess.InProcessPipelineRunner.InProcessPipelineResult

Blocks until the Pipeline execution represented by this InProcessPipelineRunner.InProcessPipelineResult is complete, returning the terminal state.

B

BACKLOG_UNKNOWN - Static variable in class com.google.cloud.dataflow.sdk.io.UnboundedSource.UnboundedReader

Constant representing an unknown amount of backlog.

baseOutputFilename - Variable in class com.google.cloud.dataflow.sdk.io.FileBasedSink

Base filename for final output files.

baseTemporaryFilename - Variable in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation

Base filename used for temporary output files.

BATCH_WORKER_HARNESS_CONTAINER_IMAGE - Static variable in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner

 

BatchBigQueryIONativeReadTranslator() - Constructor for class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner.BatchBigQueryIONativeReadTranslator

 

beforeElement(WindowedValue<T>) - Method in interface com.google.cloud.dataflow.sdk.runners.inprocess.ModelEnforcement

Called before a call to TransformEvaluator.processElement(WindowedValue) on the provided WindowedValue.

begin() - Method in class com.google.cloud.dataflow.sdk.Pipeline

Returns a PBegin owned by this Pipeline.

beginningOnDay(int) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.MonthsWindows

 

beginningOnDay(int, int) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows.YearsWindows

 

BigEndianIntegerCoder - Class in com.google.cloud.dataflow.sdk.coders

A BigEndianIntegerCoder encodes Integers in 4 bytes, big-endian.

BigEndianLongCoder - Class in com.google.cloud.dataflow.sdk.coders

A BigEndianLongCoder encodes Longs in 8 bytes, big-endian.

BigQueryIO - Class in com.google.cloud.dataflow.sdk.io

PTransforms for reading and writing BigQuery tables.

BigQueryIO.Read - Class in com.google.cloud.dataflow.sdk.io

A PTransform that reads from a BigQuery table and returns a PCollection of TableRows containing each of the rows of the table.

BigQueryIO.Read.Bound - Class in com.google.cloud.dataflow.sdk.io

A PTransform that reads from a BigQuery table and returns a bounded PCollection of TableRows.

BigQueryIO.Write - Class in com.google.cloud.dataflow.sdk.io

A PTransform that writes a PCollection containing TableRows to a BigQuery table.

BigQueryIO.Write.Bound - Class in com.google.cloud.dataflow.sdk.io

A PTransform that can write either a bounded or unbounded PCollection of TableRows to a BigQuery table.

BigQueryIO.Write.CreateDisposition - Enum in com.google.cloud.dataflow.sdk.io

An enumeration type for the BigQuery create disposition strings.

BigQueryIO.Write.WriteDisposition - Enum in com.google.cloud.dataflow.sdk.io

An enumeration type for the BigQuery write disposition strings.

BigQueryOptions - Interface in com.google.cloud.dataflow.sdk.options

Properties needed when using BigQuery with the Dataflow SDK.

BigtableIO - Class in com.google.cloud.dataflow.sdk.io.bigtable

A bounded source and sink for Google Cloud Bigtable.

BigtableIO.Read - Class in com.google.cloud.dataflow.sdk.io.bigtable

A PTransform that reads from Google Cloud Bigtable.

BigtableIO.Write - Class in com.google.cloud.dataflow.sdk.io.bigtable

A PTransform that writes to Google Cloud Bigtable.

BinaryCombineDoubleFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineDoubleFn

 

BinaryCombineFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineFn

 

BinaryCombineIntegerFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineIntegerFn

 

BinaryCombineLongFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineLongFn

 

Block() - Constructor for class com.google.cloud.dataflow.sdk.io.BlockBasedSource.Block

 

BlockBasedReader(BlockBasedSource<T>) - Constructor for class com.google.cloud.dataflow.sdk.io.BlockBasedSource.BlockBasedReader

 

BlockBasedSource<T> - Class in com.google.cloud.dataflow.sdk.io

A BlockBasedSource is a FileBasedSource where a file consists of blocks of records.

BlockBasedSource(String, long) - Constructor for class com.google.cloud.dataflow.sdk.io.BlockBasedSource

Creates a BlockBasedSource based on a file name or pattern.

BlockBasedSource(String, long, long, long) - Constructor for class com.google.cloud.dataflow.sdk.io.BlockBasedSource

Creates a BlockBasedSource for a single file.

BlockBasedSource.Block<T> - Class in com.google.cloud.dataflow.sdk.io

A Block represents a block of records that can be read.

BlockBasedSource.BlockBasedReader<T> - Class in com.google.cloud.dataflow.sdk.io

A Reader that reads records from a BlockBasedSource.

BlockingDataflowPipelineOptions - Interface in com.google.cloud.dataflow.sdk.options

Options that are used to configure the BlockingDataflowPipelineRunner.

BlockingDataflowPipelineOptions.StandardOutputFactory - Class in com.google.cloud.dataflow.sdk.options

Returns a default of System.out.

BlockingDataflowPipelineRunner - Class in com.google.cloud.dataflow.sdk.runners

A PipelineRunner that's like DataflowPipelineRunner but that waits for the launched job to finish.

BlockingDataflowPipelineRunner(DataflowPipelineRunner, BlockingDataflowPipelineOptions) - Constructor for class com.google.cloud.dataflow.sdk.runners.BlockingDataflowPipelineRunner

 

Bound() - Constructor for class com.google.cloud.dataflow.sdk.io.BigQueryIO.Write.Bound

Deprecated.

Should be private. Instead, use one of the factory methods in BigQueryIO.Write, such as BigQueryIO.Write.to(String), to create an instance of this class.

BoundedReader() - Constructor for class com.google.cloud.dataflow.sdk.io.BoundedSource.BoundedReader

 

BoundedSource<T> - Class in com.google.cloud.dataflow.sdk.io

A Source that reads a finite amount of input and, because of that, supports some additional operations.

BoundedSource() - Constructor for class com.google.cloud.dataflow.sdk.io.BoundedSource

 

BoundedSource.BoundedReader<T> - Class in com.google.cloud.dataflow.sdk.io

A Reader that reads a bounded amount of input and supports some additional operations, such as progress estimation and dynamic work rebalancing.

BoundedWindow - Class in com.google.cloud.dataflow.sdk.transforms.windowing

A BoundedWindow represents a finite grouping of elements, with an upper bound (larger timestamps represent more recent data) on the timestamps of elements that can be placed in the window.

BoundedWindow() - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.BoundedWindow

 

build() - Method in class com.google.cloud.dataflow.sdk.runners.inprocess.StepTransformResult.Builder

 

buildTemporaryFilename(String, String) - Static method in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation

Build a temporary filename using the temporary filename separator with the given prefix and suffix.

buildTrigger() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger

 

buildTrigger() - Method in interface com.google.cloud.dataflow.sdk.transforms.windowing.TriggerBuilder

Return the Trigger built by this builder.

by(PredicateT) - Static method in class com.google.cloud.dataflow.sdk.transforms.Filter

Deprecated.

use Filter.byPredicate(PredicateT), which returns a Filter transform instead of a ParDo.Bound.

byPredicate(PredicateT) - Static method in class com.google.cloud.dataflow.sdk.transforms.Filter

Returns a PTransform that takes an input PCollection<T> and returns a PCollection<T> with elements that satisfy the given predicate.

ByteArrayCoder - Class in com.google.cloud.dataflow.sdk.coders

A Coder for byte[].

ByteCoder - Class in com.google.cloud.dataflow.sdk.coders

A ByteCoder encodes Byte values in 1 byte using Java serialization.

ByteKey - Class in com.google.cloud.dataflow.sdk.io.range

A class representing a key consisting of an array of bytes.

ByteKeyRange - Class in com.google.cloud.dataflow.sdk.io.range

A class representing a range of ByteKeys.

ByteKeyRangeTracker - Class in com.google.cloud.dataflow.sdk.io.range

A RangeTracker for ByteKeys in ByteKeyRanges.

ByteStringCoder - Class in com.google.cloud.dataflow.sdk.coders

A Coder for ByteString objects based on their encoded Protocol Buffer form.

C

CalendarWindows - Class in com.google.cloud.dataflow.sdk.transforms.windowing

A collection of WindowFns that windows values into calendar-based windows such as spans of days, months, or years.

CalendarWindows() - Constructor for class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows

 

CalendarWindows.DaysWindows - Class in com.google.cloud.dataflow.sdk.transforms.windowing

A WindowFn that windows elements into periods measured by days.

CalendarWindows.MonthsWindows - Class in com.google.cloud.dataflow.sdk.transforms.windowing

A WindowFn that windows elements into periods measured by months.

CalendarWindows.YearsWindows - Class in com.google.cloud.dataflow.sdk.transforms.windowing

A WindowFn that windows elements into periods measured by years.

cancel() - Method in class com.google.cloud.dataflow.sdk.runners.DataflowPipelineJob

Cancels the job.

CannotProvideCoderException - Exception in com.google.cloud.dataflow.sdk.coders

The exception thrown when a CoderProvider cannot provide a Coder that has been requested.

CannotProvideCoderException(String) - Constructor for exception com.google.cloud.dataflow.sdk.coders.CannotProvideCoderException

 

CannotProvideCoderException(String, CannotProvideCoderException.ReasonCode) - Constructor for exception com.google.cloud.dataflow.sdk.coders.CannotProvideCoderException

 

CannotProvideCoderException(String, Throwable) - Constructor for exception com.google.cloud.dataflow.sdk.coders.CannotProvideCoderException

 

CannotProvideCoderException(String, Throwable, CannotProvideCoderException.ReasonCode) - Constructor for exception com.google.cloud.dataflow.sdk.coders.CannotProvideCoderException

 

CannotProvideCoderException(Throwable) - Constructor for exception com.google.cloud.dataflow.sdk.coders.CannotProvideCoderException

 

CannotProvideCoderException(Throwable, CannotProvideCoderException.ReasonCode) - Constructor for exception com.google.cloud.dataflow.sdk.coders.CannotProvideCoderException

 

CannotProvideCoderException.ReasonCode - Enum in com.google.cloud.dataflow.sdk.coders

Indicates the reason that Coder inference failed.

clear(Trigger<W>.TriggerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterDelayFromFirstElement

 

clear(Trigger<W>.TriggerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterPane

 

clear(Trigger<W>.TriggerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.DefaultTrigger

 

clear(Trigger<W>.TriggerContext) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger

Clear any state associated with this trigger in the given window.

clearOutputElements() - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester

Clears the record of the elements output so far to the main output.

clearSideOutputElements(TupleTag<T>) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnTester

Clears the record of the elements output so far to the side output with the given tag.

Clock - Interface in com.google.cloud.dataflow.sdk.runners.inprocess

Access to the current time.

cloneAs(Class<T>) - Method in interface com.google.cloud.dataflow.sdk.options.PipelineOptions

Makes a deep clone of this object, and transforms the cloned object into the specified type kls.

close() - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.DatastoreReader

Deprecated.

 

close() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriter

Closes the channel and return the bundle result.

close() - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource.FileBasedReader

Closes any ReadableByteChannel created for the current reader.

close() - Method in class com.google.cloud.dataflow.sdk.io.Sink.Writer

Finishes writing the bundle.

close() - Method in class com.google.cloud.dataflow.sdk.io.Source.Reader

Closes the reader.

CloudDebuggerOptions - Interface in com.google.cloud.dataflow.sdk.options

Options for controlling Cloud Debugger.

Coder<T> - Interface in com.google.cloud.dataflow.sdk.coders

A Coder<T> defines how to encode and decode values of type T into byte streams.

Coder.Context - Class in com.google.cloud.dataflow.sdk.coders

The context in which encoding or decoding is being done.

Coder.NonDeterministicException - Class in com.google.cloud.dataflow.sdk.coders

Exception thrown by Coder.verifyDeterministic() if the encoding is not deterministic, including details of why the encoding is not deterministic.

coderAllowsEncoding(Coder<T>, String) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties

 

coderConsistentWithEquals(Coder<T>, T, T) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties

 

coderConsistentWithEqualsInContext(Coder<T>, Coder.Context, T, T) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties

 

coderDecodeEncodeContentsEqual(Coder<CollectionT>, CollectionT) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties

Verifies that for the given Coder<Collection<T>>, and value of type Collection<T>, encoding followed by decoding yields an equal value of type Collection<T>, in any Coder.Context.

coderDecodeEncodeContentsEqualInContext(Coder<CollectionT>, Coder.Context, CollectionT) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties

Verifies that for the given Coder<Collection<T>>, and value of type Collection<T>, encoding followed by decoding yields an equal value of type Collection<T>, in the given Coder.Context.

coderDecodeEncodeContentsInSameOrder(Coder<IterableT>, IterableT) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties

Verifies that for the given Coder<Collection<T>>, and value of type Collection<T>, encoding followed by decoding yields an equal value of type Collection<T>, in any Coder.Context.

coderDecodeEncodeContentsInSameOrderInContext(Coder<IterableT>, Coder.Context, IterableT) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties

Verifies that for the given Coder<Iterable<T>>, and value of type Iterable<T>, encoding followed by decoding yields an equal value of type Collection<T>, in the given Coder.Context.

coderDecodeEncodeEqual(Coder<T>, T) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties

Verifies that for the given Coder<T>, and value of type T, encoding followed by decoding yields an equal value of type T, in any Coder.Context.

coderDecodeEncodeEqualInContext(Coder<T>, Coder.Context, T) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties

Verifies that for the given Coder<T>, Coder.Context, and value of type T, encoding followed by decoding yields an equal value of type T.

coderDecodesBase64(Coder<T>, String, T) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties

 

coderDecodesBase64(Coder<T>, List<String>, List<T>) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties

 

coderDecodesBase64ContentsEqual(Coder<IterableT>, String, IterableT) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties

 

coderDecodesBase64ContentsEqual(Coder<IterableT>, List<String>, List<IterableT>) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties

 

coderDeterministic(Coder<T>, T, T) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties

Verifies that for the given Coder<T>, and values of type T, if the values are equal then the encoded bytes are equal, in any Coder.Context.

coderDeterministicInContext(Coder<T>, Coder.Context, T, T) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties

Verifies that for the given Coder<T>, Coder.Context, and values of type T, if the values are equal then the encoded bytes are equal.

coderEncodesBase64(Coder<T>, T, String) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties

 

coderEncodesBase64(Coder<T>, List<T>, List<String>) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties

 

CoderException - Exception in com.google.cloud.dataflow.sdk.coders

An Exception thrown if there is a problem encoding or decoding a value.

CoderException(String) - Constructor for exception com.google.cloud.dataflow.sdk.coders.CoderException

 

CoderException(String, Throwable) - Constructor for exception com.google.cloud.dataflow.sdk.coders.CoderException

 

CoderException(Throwable) - Constructor for exception com.google.cloud.dataflow.sdk.coders.CoderException

 

CoderFactories - Class in com.google.cloud.dataflow.sdk.coders

Static utility methods for creating and working with Coders.

CoderFactory - Interface in com.google.cloud.dataflow.sdk.coders

A CoderFactory creates coders and decomposes values.

coderHasEncodingId(Coder<T>, String) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties

 

CoderProperties - Class in com.google.cloud.dataflow.sdk.testing

Properties for use in Coder tests.

CoderProperties() - Constructor for class com.google.cloud.dataflow.sdk.testing.CoderProperties

 

CoderProvider - Interface in com.google.cloud.dataflow.sdk.coders

A CoderProvider may create a Coder for any concrete class.

coderProvider() - Static method in class com.google.cloud.dataflow.sdk.coders.Proto2Coder

Deprecated.

 

coderProvider() - Static method in class com.google.cloud.dataflow.sdk.coders.protobuf.ProtoCoder

A CoderProvider that returns a ProtoCoder with an empty ExtensionRegistry.

CoderProviders - Class in com.google.cloud.dataflow.sdk.coders

Static utility methods for working with CoderProviders.

CoderRegistry - Class in com.google.cloud.dataflow.sdk.coders

A CoderRegistry allows registering the default Coder to use for a Java class, and looking up and instantiating the default Coder for a Java type.

CoderRegistry() - Constructor for class com.google.cloud.dataflow.sdk.coders.CoderRegistry

 

coderSerializable(Coder<T>) - Static method in class com.google.cloud.dataflow.sdk.testing.CoderProperties

 

CoGbkResult - Class in com.google.cloud.dataflow.sdk.transforms.join

A row result of a CoGroupByKey.

CoGbkResult(CoGbkResultSchema, Iterable<RawUnionValue>) - Constructor for class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult

A row in the PCollection resulting from a CoGroupByKey transform.

CoGbkResult(CoGbkResultSchema, Iterable<RawUnionValue>, int) - Constructor for class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult

 

CoGbkResult.CoGbkResultCoder - Class in com.google.cloud.dataflow.sdk.transforms.join

A Coder for CoGbkResults.

CoGbkResultSchema - Class in com.google.cloud.dataflow.sdk.transforms.join

A schema for the results of a CoGroupByKey.

CoGbkResultSchema(TupleTagList) - Constructor for class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResultSchema

Builds a schema from a tuple of TupleTag<?>s.

CoGroupByKey<K> - Class in com.google.cloud.dataflow.sdk.transforms.join

A PTransform that performs a CoGroupByKey on a tuple of tables.

CollectionCoder<T> - Class in com.google.cloud.dataflow.sdk.coders

A CollectionCoder encodes Collections in the format of IterableLikeCoder.

CollectionCoder(Coder<T>) - Constructor for class com.google.cloud.dataflow.sdk.coders.CollectionCoder

 

com.google.cloud.dataflow.sdk - package com.google.cloud.dataflow.sdk

Provides a simple, powerful model for building both batch and streaming parallel data processing Pipelines.

com.google.cloud.dataflow.sdk.annotations - package com.google.cloud.dataflow.sdk.annotations

Defines annotations used across the SDK.

com.google.cloud.dataflow.sdk.coders - package com.google.cloud.dataflow.sdk.coders

Defines Coders to specify how data is encoded to and decoded from byte strings.

com.google.cloud.dataflow.sdk.coders.protobuf - package com.google.cloud.dataflow.sdk.coders.protobuf

Defines a Coder for Protocol Buffers messages, ProtoCoder.

com.google.cloud.dataflow.sdk.io - package com.google.cloud.dataflow.sdk.io

Defines transforms for reading and writing common storage formats, including AvroIO, BigQueryIO, and TextIO.

com.google.cloud.dataflow.sdk.io.bigtable - package com.google.cloud.dataflow.sdk.io.bigtable

Defines transforms for reading and writing from Google Cloud Bigtable.

com.google.cloud.dataflow.sdk.io.datastore - package com.google.cloud.dataflow.sdk.io.datastore

Provides an API for reading from and writing to Google Cloud Datastore over different versions of the Cloud Datastore Client libraries.

com.google.cloud.dataflow.sdk.io.range - package com.google.cloud.dataflow.sdk.io.range

Provides thread-safe helpers for implementing dynamic work rebalancing in position-based bounded sources.

com.google.cloud.dataflow.sdk.options - package com.google.cloud.dataflow.sdk.options

Defines PipelineOptions for configuring pipeline execution.

com.google.cloud.dataflow.sdk.runners - package com.google.cloud.dataflow.sdk.runners

Defines runners for executing Pipelines in different modes, including DirectPipelineRunner and DataflowPipelineRunner.

com.google.cloud.dataflow.sdk.runners.inprocess - package com.google.cloud.dataflow.sdk.runners.inprocess

Defines the InProcessPipelineRunner, which executes both Bounded and Unbounded Pipelines on the local machine.

com.google.cloud.dataflow.sdk.testing - package com.google.cloud.dataflow.sdk.testing

Defines utilities for unit testing Dataflow pipelines.

com.google.cloud.dataflow.sdk.transforms - package com.google.cloud.dataflow.sdk.transforms

Defines PTransforms for transforming data in a pipeline.

com.google.cloud.dataflow.sdk.transforms.display - package com.google.cloud.dataflow.sdk.transforms.display

Defines HasDisplayData for annotating components which provide display data used within UIs and diagnostic tools.

com.google.cloud.dataflow.sdk.transforms.join - package com.google.cloud.dataflow.sdk.transforms.join

Defines the CoGroupByKey transform for joining multiple PCollections.

com.google.cloud.dataflow.sdk.transforms.windowing - package com.google.cloud.dataflow.sdk.transforms.windowing

Defines the Window transform for dividing the elements in a PCollection into windows, and the Trigger for controlling when those elements are output.

com.google.cloud.dataflow.sdk.values - package com.google.cloud.dataflow.sdk.values

Defines PCollection and other classes for representing data in a Pipeline.

Combine - Class in com.google.cloud.dataflow.sdk.transforms

PTransforms for combining PCollection elements globally and per-key.

combine(Instant, Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFn

Combines the given output times, which must be from the same window, into an output time for a computed value.

combine(Instant, Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFn.Defaults

Combines the given output times, which must be from the same window, into an output time for a computed value.

combine(Instant, Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFn.DependsOnlyOnWindow

Combines the given output times, which must be from the same window, into an output time for a computed value.

Combine.AccumulatingCombineFn<InputT,AccumT extends Combine.AccumulatingCombineFn.Accumulator<InputT,AccumT,OutputT>,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms

A CombineFn that uses a subclass of Combine.AccumulatingCombineFn.Accumulator as its accumulator type.

Combine.AccumulatingCombineFn.Accumulator<InputT,AccumT,OutputT> - Interface in com.google.cloud.dataflow.sdk.transforms

The type of mutable accumulator values used by this AccumulatingCombineFn.

Combine.BinaryCombineDoubleFn - Class in com.google.cloud.dataflow.sdk.transforms

An abstract subclass of Combine.CombineFn for implementing combiners that are more easily and efficiently expressed as binary operations on doubles.

Combine.BinaryCombineFn<V> - Class in com.google.cloud.dataflow.sdk.transforms

An abstract subclass of Combine.CombineFn for implementing combiners that are more easily expressed as binary operations.

Combine.BinaryCombineIntegerFn - Class in com.google.cloud.dataflow.sdk.transforms

An abstract subclass of Combine.CombineFn for implementing combiners that are more easily and efficiently expressed as binary operations on ints

Combine.BinaryCombineLongFn - Class in com.google.cloud.dataflow.sdk.transforms

An abstract subclass of Combine.CombineFn for implementing combiners that are more easily and efficiently expressed as binary operations on longs.

Combine.CombineFn<InputT,AccumT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms

A CombineFn<InputT, AccumT, OutputT> specifies how to combine a collection of input values of type InputT into a single output value of type OutputT.

Combine.Globally<InputT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms

Combine.Globally<InputT, OutputT> takes a PCollection<InputT> and returns a PCollection<OutputT> whose elements are the result of combining all the elements in each window of the input PCollection, using a specified CombineFn<InputT, AccumT, OutputT>.

Combine.GloballyAsSingletonView<InputT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms

Combine.GloballyAsSingletonView<InputT, OutputT> takes a PCollection<InputT> and returns a PCollectionView<OutputT> whose elements are the result of combining all the elements in each window of the input PCollection, using a specified CombineFn<InputT, AccumT, OutputT>.

Combine.GroupedValues<K,InputT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms

GroupedValues<K, InputT, OutputT> takes a PCollection<KV<K, Iterable<InputT>>>, such as the result of GroupByKey, applies a specified KeyedCombineFn<K, InputT, AccumT, OutputT> to each of the input KV<K, Iterable<InputT>> elements to produce a combined output KV<K, OutputT> element, and returns a PCollection<KV<K, OutputT>> containing all the combined output elements.

Combine.Holder<V> - Class in com.google.cloud.dataflow.sdk.transforms

Holds a single value value of type V which may or may not be present.

Combine.IterableCombineFn<V> - Class in com.google.cloud.dataflow.sdk.transforms

Converts a SerializableFunction from Iterable<V>s to Vs into a simple Combine.CombineFn over Vs.

Combine.KeyedCombineFn<K,InputT,AccumT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms

A KeyedCombineFn<K, InputT, AccumT, OutputT> specifies how to combine a collection of input values of type InputT, associated with a key of type K, into a single output value of type OutputT.

Combine.PerKey<K,InputT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms

PerKey<K, InputT, OutputT> takes a PCollection<KV<K, InputT>>, groups it by key, applies a combining function to the InputT values associated with each key to produce a combined OutputT value, and returns a PCollection<KV<K, OutputT>> representing a map from each distinct key of the input PCollection to the corresponding combined value.

Combine.PerKeyWithHotKeyFanout<K,InputT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms

Like Combine.PerKey, but sharding the combining of hot keys.

Combine.SimpleCombineFn<V> - Class in com.google.cloud.dataflow.sdk.transforms

Deprecated.

CombineFn() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Combine.CombineFn

 

CombineFnBase - Class in com.google.cloud.dataflow.sdk.transforms

This class contains the shared interfaces and abstract classes for different types of combine functions.

CombineFnBase() - Constructor for class com.google.cloud.dataflow.sdk.transforms.CombineFnBase

 

CombineFnBase.GlobalCombineFn<InputT,AccumT,OutputT> - Interface in com.google.cloud.dataflow.sdk.transforms

A GloballyCombineFn<InputT, AccumT, OutputT> specifies how to combine a collection of input values of type InputT into a single output value of type OutputT.

CombineFnBase.PerKeyCombineFn<K,InputT,AccumT,OutputT> - Interface in com.google.cloud.dataflow.sdk.transforms

A PerKeyCombineFn<K, InputT, AccumT, OutputT> specifies how to combine a collection of input values of type InputT, associated with a key of type K, into a single output value of type OutputT.

CombineFns - Class in com.google.cloud.dataflow.sdk.transforms

Static utility methods that create combine function instances.

CombineFns() - Constructor for class com.google.cloud.dataflow.sdk.transforms.CombineFns

 

CombineFns.CoCombineResult - Class in com.google.cloud.dataflow.sdk.transforms

A tuple of outputs produced by a composed combine functions.

CombineFns.ComposeCombineFnBuilder - Class in com.google.cloud.dataflow.sdk.transforms

A builder class to construct a composed CombineFnBase.GlobalCombineFn.

CombineFns.ComposedCombineFn<DataT> - Class in com.google.cloud.dataflow.sdk.transforms

A composed Combine.CombineFn that applies multiple CombineFns.

CombineFns.ComposedCombineFnWithContext<DataT> - Class in com.google.cloud.dataflow.sdk.transforms

A composed CombineWithContext.CombineFnWithContext that applies multiple CombineFnWithContexts.

CombineFns.ComposedKeyedCombineFn<DataT,K> - Class in com.google.cloud.dataflow.sdk.transforms

A composed Combine.KeyedCombineFn that applies multiple KeyedCombineFns.

CombineFns.ComposedKeyedCombineFnWithContext<DataT,K> - Class in com.google.cloud.dataflow.sdk.transforms

A composed CombineWithContext.KeyedCombineFnWithContext that applies multiple KeyedCombineFnWithContexts.

CombineFns.ComposeKeyedCombineFnBuilder - Class in com.google.cloud.dataflow.sdk.transforms

A builder class to construct a composed CombineFnBase.PerKeyCombineFn.

CombineFnWithContext() - Constructor for class com.google.cloud.dataflow.sdk.transforms.CombineWithContext.CombineFnWithContext

 

combineOutputTimes(OutputTimeFn<?>, Iterable<? extends Instant>) - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.OutputTimeFns

Applies the given OutputTimeFn to the given output times, obtaining the output time for a value computed.

CombineWithContext - Class in com.google.cloud.dataflow.sdk.transforms

This class contains combine functions that have access to PipelineOptions and side inputs through CombineWithContext.Context.

CombineWithContext() - Constructor for class com.google.cloud.dataflow.sdk.transforms.CombineWithContext

 

CombineWithContext.CombineFnWithContext<InputT,AccumT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms

A combine function that has access to PipelineOptions and side inputs through CombineWithContext.Context.

CombineWithContext.Context - Class in com.google.cloud.dataflow.sdk.transforms

Information accessible to all methods in CombineFnWithContext and KeyedCombineFnWithContext.

CombineWithContext.KeyedCombineFnWithContext<K,InputT,AccumT,OutputT> - Class in com.google.cloud.dataflow.sdk.transforms

A keyed combine function that has access to PipelineOptions and side inputs through CombineWithContext.Context.

CombineWithContext.RequiresContextInternal - Interface in com.google.cloud.dataflow.sdk.transforms

An internal interface for signaling that a GloballyCombineFn or a PerKeyCombineFn needs to access CombineWithContext.Context.

commit(Instant) - Method in interface com.google.cloud.dataflow.sdk.runners.inprocess.InProcessPipelineRunner.UncommittedBundle

Commits this InProcessPipelineRunner.UncommittedBundle, returning an immutable InProcessPipelineRunner.CommittedBundle containing all of the elements that were added to it.

compact(AccumT) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.CombineFn

Returns an accumulator that represents the same logical value as the input accumulator, but may have a more compact representation.

compact(List<V>) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.IterableCombineFn

 

compact(K, AccumT) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.KeyedCombineFn

Returns an accumulator that represents the same logical value as the input accumulator, but may have a more compact representation.

compact(Object[]) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedCombineFn

 

compact(Object[], CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedCombineFnWithContext

 

compact(K, Object[]) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedKeyedCombineFn

 

compact(K, Object[], CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedKeyedCombineFnWithContext

 

compact(AccumT, CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineWithContext.CombineFnWithContext

Returns an accumulator that represents the same logical value as the input accumulator, but may have a more compact representation.

compact(K, AccumT, CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineWithContext.KeyedCombineFnWithContext

Returns an accumulator that represents the same logical value as the input accumulator, but may have a more compact representation.

compare(T, T) - Method in class com.google.cloud.dataflow.sdk.transforms.Top.Largest

 

compare(T, T) - Method in class com.google.cloud.dataflow.sdk.transforms.Top.Smallest

 

compare(KV<K, V>, KV<K, V>) - Method in class com.google.cloud.dataflow.sdk.values.KV.OrderByKey

 

compare(KV<K, V>, KV<K, V>) - Method in class com.google.cloud.dataflow.sdk.values.KV.OrderByValue

 

compareTo(ByteKey) - Method in class com.google.cloud.dataflow.sdk.io.range.ByteKey

ByteKey implements Comparable<ByteKey> by comparing the arrays in lexicographic order.

compareTo(IntervalWindow) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.IntervalWindow

 

compose() - Static method in class com.google.cloud.dataflow.sdk.transforms.CombineFns

Returns a CombineFns.ComposeCombineFnBuilder to construct a composed CombineFnBase.GlobalCombineFn.

ComposeCombineFnBuilder() - Constructor for class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposeCombineFnBuilder

 

composeKeyed() - Static method in class com.google.cloud.dataflow.sdk.transforms.CombineFns

Returns a CombineFns.ComposeKeyedCombineFnBuilder to construct a composed CombineFnBase.PerKeyCombineFn.

ComposeKeyedCombineFnBuilder() - Constructor for class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposeKeyedCombineFnBuilder

 

CompressedReader(CompressedSource<T>, FileBasedSource.FileBasedReader<T>) - Constructor for class com.google.cloud.dataflow.sdk.io.CompressedSource.CompressedReader

Create a CompressedReader from a CompressedSource and delegate reader.

CompressedSource<T> - Class in com.google.cloud.dataflow.sdk.io

A Source that reads from compressed files.

CompressedSource.CompressedReader<T> - Class in com.google.cloud.dataflow.sdk.io

Reader for a CompressedSource.

CompressedSource.CompressionMode - Enum in com.google.cloud.dataflow.sdk.io

Default compression types supported by the CompressedSource.

CompressedSource.DecompressingChannelFactory - Interface in com.google.cloud.dataflow.sdk.io

Factory interface for creating channels that decompress the content of an underlying channel.

computeTargetTimestamp(Instant) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.AfterDelayFromFirstElement

 

consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.BigEndianIntegerCoder

Returns true if this Coder is injective with respect to Object.equals(java.lang.Object).

consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.BigEndianLongCoder

Returns true if this Coder is injective with respect to Object.equals(java.lang.Object).

consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.ByteCoder

Returns true if this Coder is injective with respect to Object.equals(java.lang.Object).

consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.ByteStringCoder

Returns true if this Coder is injective with respect to Object.equals(java.lang.Object).

consistentWithEquals() - Method in interface com.google.cloud.dataflow.sdk.coders.Coder

Returns true if this Coder is injective with respect to Object.equals(java.lang.Object).

consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.DoubleCoder

Returns true if this Coder is injective with respect to Object.equals(java.lang.Object).

consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.DurationCoder

Returns true if this Coder is injective with respect to Object.equals(java.lang.Object).

consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.InstantCoder

Returns true if this Coder is injective with respect to Object.equals(java.lang.Object).

consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.KvCoder

 

consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.NullableCoder

NullableCoder is consistent with equals if the nested Coder is.

consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.StandardCoder

Returns true if this Coder is injective with respect to Object.equals(java.lang.Object).

consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.StringUtf8Coder

Returns true if this Coder is injective with respect to Object.equals(java.lang.Object).

consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.VarIntCoder

Returns true if this Coder is injective with respect to Object.equals(java.lang.Object).

consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.VarLongCoder

Returns true if this Coder is injective with respect to Object.equals(java.lang.Object).

consistentWithEquals() - Method in class com.google.cloud.dataflow.sdk.coders.VoidCoder

Returns true if this Coder is injective with respect to Object.equals(java.lang.Object).

contains(IntervalWindow) - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.IntervalWindow

Returns whether this window contains the given window.

containsInAnyOrder(T...) - Method in interface com.google.cloud.dataflow.sdk.testing.DataflowAssert.IterableAssert

Asserts that the iterable in question contains the provided elements.

containsInAnyOrder(Iterable<T>) - Method in interface com.google.cloud.dataflow.sdk.testing.DataflowAssert.IterableAssert

Asserts that the iterable in question contains the provided elements.

containsKey(ByteKey) - Method in class com.google.cloud.dataflow.sdk.io.range.ByteKeyRange

Returns true if the specified ByteKey is contained within this range.

Context(boolean) - Constructor for class com.google.cloud.dataflow.sdk.coders.Coder.Context

 

Context() - Constructor for class com.google.cloud.dataflow.sdk.transforms.CombineWithContext.Context

 

Context() - Constructor for class com.google.cloud.dataflow.sdk.transforms.DoFn.Context

 

Context() - Constructor for class com.google.cloud.dataflow.sdk.transforms.DoFnWithContext.Context

 

copyFrom(byte[]) - Static method in class com.google.cloud.dataflow.sdk.io.range.ByteKey

Creates a new ByteKey backed by a copy of the specified byte[].

copyToOutputFiles(List<String>, PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation

Copy temporary files to final output filenames using the file naming template.

Count - Class in com.google.cloud.dataflow.sdk.transforms

PTransorms to count the elements in a PCollection.

Count.PerElement<T> - Class in com.google.cloud.dataflow.sdk.transforms

Count.PerElement<T> takes a PCollection<T> and returns a PCollection<KV<T, Long>> representing a map from each distinct element of the input PCollection to the number of times that element occurs in the input.

CounterMark(long, Instant) - Constructor for class com.google.cloud.dataflow.sdk.io.CountingSource.CounterMark

Creates a checkpoint mark reflecting the last emitted value.

CountingInput - Class in com.google.cloud.dataflow.sdk.io

A PTransform that produces longs.

CountingInput() - Constructor for class com.google.cloud.dataflow.sdk.io.CountingInput

 

CountingInput.BoundedCountingInput - Class in com.google.cloud.dataflow.sdk.io

A PTransform that will produce a specified number of Longs starting from 0.

CountingInput.UnboundedCountingInput - Class in com.google.cloud.dataflow.sdk.io

A PTransform that will produce numbers starting from 0 up to Long.MAX_VALUE.

CountingSource - Class in com.google.cloud.dataflow.sdk.io

A source that produces longs.

CountingSource.CounterMark - Class in com.google.cloud.dataflow.sdk.io

The checkpoint for an unbounded CountingSource is simply the last value produced.

create(List<? extends Coder<?>>) - Method in interface com.google.cloud.dataflow.sdk.coders.CoderFactory

Returns a Coder<?>, given argument coder to use for values of a particular type, given the Coders for each of the type's generic parameter types.

create(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.options.BlockingDataflowPipelineOptions.StandardOutputFactory

 

create(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions.DataflowClientFactory

 

create(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions.PathValidatorFactory

 

create(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions.StagerFactory

 

create(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.options.DataflowPipelineOptions.JobNameFactory

 

create(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.options.DataflowPipelineWorkerPoolOptions.WorkerHarnessContainerImageFactory

 

create(PipelineOptions) - Method in interface com.google.cloud.dataflow.sdk.options.DefaultValueFactory

Creates a default value for a getter marked with Default.InstanceFactory.

create(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.options.GcpOptions.CredentialDirFactory

 

create(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.options.GcpOptions.DefaultProjectFactory

 

create(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.options.GcpOptions.GcpUserCredentialsFactory

 

create(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.options.GcsOptions.ExecutorServiceFactory

 

create(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.options.PipelineOptions.AtomicLongFactory

 

create() - Method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory.Builder

Creates and returns an object that implements PipelineOptions using the values configured on this builder during construction.

create() - Static method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory

Creates and returns an object that implements PipelineOptions.

create(PipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.Pipeline

Constructs a pipeline from the provided options.

create(DataflowPipelineOptions) - Static method in class com.google.cloud.dataflow.sdk.runners.DataflowPipeline

Creates and returns a new DataflowPipeline instance for tests.

create(Combine.GroupedValues<K, InputT, OutputT>, PCollection<KV<K, Iterable<InputT>>>, boolean, Random) - Static method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.TestCombineDoFn

 

create() - Method in interface com.google.cloud.dataflow.sdk.runners.inprocess.ExecutorServiceFactory

Create a new ExecutorService.

create() - Static method in class com.google.cloud.dataflow.sdk.runners.inprocess.NanosOffsetClock

 

create() - Static method in class com.google.cloud.dataflow.sdk.testing.TestPipeline

Creates and returns a new test pipeline.

create(Coder<T>) - Static method in class com.google.cloud.dataflow.sdk.testing.TestStream

Create a new TestStream.Builder with no elements and watermark equal to BoundedWindow.TIMESTAMP_MIN_VALUE.

create(int, ComparatorT) - Static method in class com.google.cloud.dataflow.sdk.transforms.ApproximateQuantiles.ApproximateQuantilesCombineFn

Returns an approximate quantiles combiner with the given compareFn and desired number of quantiles.

create(int) - Static method in class com.google.cloud.dataflow.sdk.transforms.ApproximateQuantiles.ApproximateQuantilesCombineFn

Like ApproximateQuantiles.ApproximateQuantilesCombineFn.create(int, Comparator), but sorts values using their natural ordering.

create(int, ComparatorT, long, double) - Static method in class com.google.cloud.dataflow.sdk.transforms.ApproximateQuantiles.ApproximateQuantilesCombineFn

Creates an approximate quantiles combiner with the given compareFn and desired number of quantiles.

Create<T> - Class in com.google.cloud.dataflow.sdk.transforms

Create<T> takes a collection of elements of type T known when the pipeline is constructed and returns a PCollection<T> containing the elements.

Create() - Constructor for class com.google.cloud.dataflow.sdk.transforms.Create

 

create() - Static method in class com.google.cloud.dataflow.sdk.transforms.GroupByKey

Returns a GroupByKey<K, V> PTransform.

create() - Static method in class com.google.cloud.dataflow.sdk.transforms.join.CoGroupByKey

Returns a CoGroupByKey<K> PTransform.

create() - Static method in class com.google.cloud.dataflow.sdk.transforms.Keys

Returns a Keys<K> PTransform.

create() - Static method in class com.google.cloud.dataflow.sdk.transforms.KvSwap

Returns a KvSwap<K, V> PTransform.

create() - Static method in class com.google.cloud.dataflow.sdk.transforms.RemoveDuplicates

Returns a RemoveDuplicates<T> PTransform.

create() - Static method in class com.google.cloud.dataflow.sdk.transforms.Values

Returns a Values<V> PTransform.

Create.TimestampedValues<T> - Class in com.google.cloud.dataflow.sdk.transforms

A PTransform that creates a PCollection whose elements have associated timestamps.

Create.Values<T> - Class in com.google.cloud.dataflow.sdk.transforms

A PTransform that creates a PCollection from a set of in-memory objects.

createAccumulator() - Method in class com.google.cloud.dataflow.sdk.transforms.ApproximateQuantiles.ApproximateQuantilesCombineFn

 

createAccumulator() - Method in class com.google.cloud.dataflow.sdk.transforms.ApproximateUnique.ApproximateUniqueCombineFn

 

createAccumulator() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineDoubleFn

 

createAccumulator() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineFn

 

createAccumulator() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineIntegerFn

 

createAccumulator() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.BinaryCombineLongFn

 

createAccumulator() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.CombineFn

Returns a new, mutable accumulator value, representing the accumulation of zero input values.

createAccumulator() - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.IterableCombineFn

 

createAccumulator(K) - Method in class com.google.cloud.dataflow.sdk.transforms.Combine.KeyedCombineFn

Returns a new, mutable accumulator value representing the accumulation of zero input values.

createAccumulator() - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedCombineFn

 

createAccumulator(CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedCombineFnWithContext

 

createAccumulator(K) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedKeyedCombineFn

 

createAccumulator(K, CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineFns.ComposedKeyedCombineFnWithContext

 

createAccumulator(CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineWithContext.CombineFnWithContext

Returns a new, mutable accumulator value, representing the accumulation of zero input values.

createAccumulator(K, CombineWithContext.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.CombineWithContext.KeyedCombineFnWithContext

Returns a new, mutable accumulator value representing the accumulation of zero input values.

createAccumulator() - Method in class com.google.cloud.dataflow.sdk.transforms.Sample.FixedSizedSampleFn

 

createAccumulator() - Method in class com.google.cloud.dataflow.sdk.transforms.Top.TopCombineFn

 

createAggregator(String, Combine.CombineFn<? super AggInputT, ?, AggOutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn

Returns an Aggregator with aggregation logic specified by the Combine.CombineFn argument.

createAggregator(String, SerializableFunction<Iterable<AggInputT>, AggInputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn

Returns an Aggregator with the aggregation logic specified by the SerializableFunction argument.

createAggregator(String, Combine.CombineFn<? super AggInputT, ?, AggOutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnWithContext

Returns an Aggregator with aggregation logic specified by the Combine.CombineFn argument.

createAggregator(String, SerializableFunction<Iterable<AggInputT>, AggInputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFnWithContext

Returns an Aggregator with the aggregation logic specified by the SerializableFunction argument.

createAggregatorInternal(String, Combine.CombineFn<AggInputT, ?, AggOutputT>) - Method in class com.google.cloud.dataflow.sdk.transforms.DoFn.Context

Creates an Aggregator in the DoFn context with the specified name and aggregation logic specified by Combine.CombineFn.

createContextual(DeserializationContext, BeanProperty) - Method in class com.google.cloud.dataflow.sdk.options.ValueProvider.Deserializer

 

createDatumReader() - Method in class com.google.cloud.dataflow.sdk.coders.AvroCoder

Deprecated.

For AvroCoder internal use only.

createDatumWriter() - Method in class com.google.cloud.dataflow.sdk.coders.AvroCoder

Deprecated.

For AvroCoder internal use only.

createDecompressingChannel(ReadableByteChannel) - Method in enum com.google.cloud.dataflow.sdk.io.CompressedSource.CompressionMode

 

createDecompressingChannel(ReadableByteChannel) - Method in interface com.google.cloud.dataflow.sdk.io.CompressedSource.DecompressingChannelFactory

Given a channel, create a channel that decompresses the content read from the channel.

createForSubrangeOfFile(String, long, long) - Method in class com.google.cloud.dataflow.sdk.io.AvroSource

 

createForSubrangeOfFile(String, long, long) - Method in class com.google.cloud.dataflow.sdk.io.BlockBasedSource

Creates a BlockBasedSource for the specified range in a single file.

createForSubrangeOfFile(String, long, long) - Method in class com.google.cloud.dataflow.sdk.io.CompressedSource

Creates a CompressedSource for a subrange of a file.

createForSubrangeOfFile(String, long, long) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource

Creates and returns a new FileBasedSource of the same type as the current FileBasedSource backed by a given file and an offset range.

createForSubrangeOfFile(String, long, long) - Method in class com.google.cloud.dataflow.sdk.io.XmlSource

 

createForTest() - Static method in class com.google.cloud.dataflow.sdk.runners.DirectPipeline

Creates and returns a new DirectPipeline instance for tests.

createForTest() - Static method in class com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner

Constructs a runner with default properties for testing.

createFromSystemProperties() - Static method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory

Deprecated.

for internal use only

createFromSystemPropertiesInternal() - Static method in class com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory

Creates a set of Dataflow worker harness options based of a set of known system properties.

createPane(boolean, boolean, PaneInfo.Timing) - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.PaneInfo

 

createPane(boolean, boolean, PaneInfo.Timing, long, long) - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.PaneInfo

Factory method to create a PaneInfo with the specified parameters.

createPrimitiveOutputInternal(Pipeline, WindowingStrategy<?, ?>, PCollection.IsBounded) - Static method in class com.google.cloud.dataflow.sdk.values.PCollection

Creates and returns a new PCollection for a primitive output.

createReader(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.BoundedSource

Returns a new BoundedSource.BoundedReader that reads from this source.

createReader(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Source

Deprecated.

 

createReader(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource

 

createReader(PipelineOptions, CheckpointMarkT) - Method in class com.google.cloud.dataflow.sdk.io.UnboundedSource

Create a new UnboundedSource.UnboundedReader to read from this source, resuming from the given checkpoint if present.

createSingleFileReader(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.AvroSource

 

createSingleFileReader(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.BlockBasedSource

Creates a BlockBasedReader.

createSingleFileReader(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.CompressedSource

Creates a FileBasedReader to read a single file.

createSingleFileReader(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource

Creates and returns an instance of a FileBasedReader implementation for the current source assuming the source represents a single file.

createSingleFileReader(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.XmlSource

 

createSourceForSubrange(long, long) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSource

 

createSourceForSubrange(long, long) - Method in class com.google.cloud.dataflow.sdk.io.OffsetBasedSource

Returns an OffsetBasedSource for a subrange of the current source.

createStructuralValues(Coder<T>, List<T>) - Static method in class com.google.cloud.dataflow.sdk.testing.SourceTestUtils

Testing utilities below depend on standard assertions and matchers to compare elements read by sources.

createWriteOperation(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.DatastoreIO.Sink

Deprecated.

 

createWriteOperation(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink

Return a subclass of FileBasedSink.FileBasedWriteOperation that will manage the write to the sink.

createWriteOperation(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.Sink

Returns an instance of a Sink.WriteOperation that can write to this Sink.

createWriteOperation(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.XmlSink.Bound

Creates an XmlSink.XmlWriteOperation.

createWriter(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.FileBasedSink.FileBasedWriteOperation

Clients must implement to return a subclass of FileBasedSink.FileBasedWriter.

createWriter(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.Sink.WriteOperation

Creates a new Sink.Writer to write a bundle of the input to the sink.

createWriter(PipelineOptions) - Method in class com.google.cloud.dataflow.sdk.io.XmlSink.XmlWriteOperation

Creates a XmlSink.XmlWriter with a marshaller for the type it will write.

CredentialDirFactory() - Constructor for class com.google.cloud.dataflow.sdk.options.GcpOptions.CredentialDirFactory

 

currentEventTime() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerContext

The current event time for the input or null if unknown.

currentProcessingTime() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerContext

The current processing time.

currentSynchronizedProcessingTime() - Method in class com.google.cloud.dataflow.sdk.transforms.windowing.Trigger.TriggerContext

The current synchronized upstream processing time or null if unknown.

CustomCoder<T> - Class in com.google.cloud.dataflow.sdk.coders

An abstract base class for writing a Coder class that encodes itself via Java serialization.

CustomCoder() - Constructor for class com.google.cloud.dataflow.sdk.coders.CustomCoder

 

D

DataflowAssert - Class in com.google.cloud.dataflow.sdk.testing

An assertion on the contents of a PCollection incorporated into the pipeline.

DataflowAssert.GroupThenAssert<T> - Class in com.google.cloud.dataflow.sdk.testing

A transform that applies an assertion-checking function over iterables of ActualT to the entirety of the contents of its input.

DataflowAssert.GroupThenAssertForSingleton<T> - Class in com.google.cloud.dataflow.sdk.testing

A transform that applies an assertion-checking function to a single iterable contained as the sole element of a PCollection.

DataflowAssert.IterableAssert<T> - Interface in com.google.cloud.dataflow.sdk.testing

Builder interface for assertions applicable to iterables and PCollection contents.

DataflowAssert.OneSideInputAssert<ActualT> - Class in com.google.cloud.dataflow.sdk.testing

An assertion checker that takes a single PCollectionView<ActualT> and an assertion over ActualT, and checks it within a Beam pipeline.

DataflowAssert.SingletonAssert<T> - Interface in com.google.cloud.dataflow.sdk.testing

Builder interface for assertions applicable to a single value.

DataflowClientFactory() - Constructor for class com.google.cloud.dataflow.sdk.options.DataflowPipelineDebugOptions.DataflowClientFactory

 

DataflowJobAlreadyExistsException - Exception in com.google.cloud.dataflow.sdk.runners

An exception that is thrown if the unique job name constraint of the Dataflow service is broken because an existing job with the same job name is currently active.

DataflowJobAlreadyExistsException(DataflowPipelineJob, String) - Constructor for exception com.google.cloud.dataflow.sdk.runners.DataflowJobAlreadyExistsException

Create a new DataflowJobAlreadyExistsException with the specified DataflowPipelineJob and message.

DataflowJobAlreadyUpdatedException - Exception in com.google.cloud.dataflow.sdk.runners

An exception that is thrown if the existing job has already been updated within the Dataflow service and is no longer able to be updated.

DataflowJobAlreadyUpdatedException(DataflowPipelineJob, String) - Constructor for exception com.google.cloud.dataflow.sdk.runners.DataflowJobAlreadyUpdatedException

Create a new DataflowJobAlreadyUpdatedException with the specified DataflowPipelineJob and message.

DataflowJobCancelledException - Exception in com.google.cloud.dataflow.sdk.runners

Signals that a job run by a BlockingDataflowPipelineRunner was updated during execution.

DataflowJobCancelledException(DataflowPipelineJob, String) - Constructor for exception com.google.cloud.dataflow.sdk.runners.DataflowJobCancelledException

Create a new DataflowJobAlreadyUpdatedException with the specified DataflowPipelineJob and message.

DataflowJobCancelledException(DataflowPipelineJob, String, Throwable) - Constructor for exception com.google.cloud.dataflow.sdk.runners.DataflowJobCancelledException

Create a new DataflowJobAlreadyUpdatedException with the specified DataflowPipelineJob, message, and cause.

DataflowJobException - Exception in com.google.cloud.dataflow.sdk.runners

A RuntimeException that contains information about a DataflowPipelineJob.

DataflowJobExecutionException - Exception in com.google.cloud.dataflow.sdk.runners

Signals that a job run by a BlockingDataflowPipelineRunner fails during execution, and provides access to the failed job.

DataflowJobUpdatedException - Exception in com.google.cloud.dataflow.sdk.runners

Signals that a job run by a BlockingDataflowPipelineRunner was updated during execution.

DataflowJobUpdatedException(DataflowPipelineJob, String, DataflowPipelineJob) - Constructor for exception com.google.cloud.dataflow.sdk.runners.DataflowJobUpdatedException

Create a new DataflowJobUpdatedException with the specified original DataflowPipelineJob, message, and replacement DataflowPipelineJob.

DataflowJobUpdatedException(DataflowPipelineJob, String, DataflowPipelineJob, Throwable) - Constructor for exception com.google.cloud.dataflow.sdk.runners.DataflowJobUpdatedException

Create a new DataflowJobUpdatedException with the specified original DataflowPipelineJob, message, replacement DataflowPipelineJob, and cause.

DataflowPipeline - Class in com.google.cloud.dataflow.sdk.runners

A DataflowPipeline is a Pipeline that returns a DataflowPipelineJob when it is Pipeline.run().

DataflowPipelineDebugOptions - Interface in com.google.cloud.dataflow.sdk.options

Internal.

DataflowPipelineDebugOptions.DataflowClientFactory - Class in com.google.cloud.dataflow.sdk.options

Returns the default Dataflow client built from the passed in PipelineOptions.

DataflowPipelineDebugOptions.PathValidatorFactory - Class in com.google.cloud.dataflow.sdk.options

Creates a PathValidator object using the class specified in DataflowPipelineDebugOptions.getPathValidatorClass().

DataflowPipelineDebugOptions.StagerFactory - Class in com.google.cloud.dataflow.sdk.options

Creates a Stager object using the class specified in DataflowPipelineDebugOptions.getStagerClass().

DataflowPipelineJob - Class in com.google.cloud.dataflow.sdk.runners

A DataflowPipelineJob represents a job submitted to Dataflow using DataflowPipelineRunner.

DataflowPipelineJob(String, String, Dataflow, DataflowAggregatorTransforms) - Constructor for class com.google.cloud.dataflow.sdk.runners.DataflowPipelineJob

Constructs the job.

DataflowPipelineOptions - Interface in com.google.cloud.dataflow.sdk.options

Options that can be used to configure the DataflowPipeline.

DataflowPipelineOptions.JobNameFactory - Class in com.google.cloud.dataflow.sdk.options

Returns a normalized job name constructed from ApplicationNameOptions.getAppName(), the local system user name (if available), and the current time.

DataflowPipelineRegistrar - Class in com.google.cloud.dataflow.sdk.runners

Contains the PipelineOptionsRegistrar and PipelineRunnerRegistrar for the DataflowPipeline.

DataflowPipelineRegistrar.Options - Class in com.google.cloud.dataflow.sdk.runners

Register the DataflowPipelineOptions and BlockingDataflowPipelineOptions.

DataflowPipelineRegistrar.Runner - Class in com.google.cloud.dataflow.sdk.runners

Register the DataflowPipelineRunner and BlockingDataflowPipelineRunner.

DataflowPipelineRunner - Class in com.google.cloud.dataflow.sdk.runners

A PipelineRunner that executes the operations in the pipeline by first translating them to the Dataflow representation using the DataflowPipelineTranslator and then submitting them to a Dataflow service for execution.

DataflowPipelineRunner(DataflowPipelineOptions) - Constructor for class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner

 

DataflowPipelineRunner.BatchBigQueryIONativeReadTranslator - Class in com.google.cloud.dataflow.sdk.runners

Implements BigQueryIO Read translation for the Dataflow backend.

DataflowPipelineRunnerHooks - Class in com.google.cloud.dataflow.sdk.runners

An instance of this class can be passed to the DataflowPipelineRunner to add user defined hooks to be invoked at various times during pipeline execution.

DataflowPipelineRunnerHooks() - Constructor for class com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunnerHooks

 

DataflowPipelineTranslator - Class in com.google.cloud.dataflow.sdk.runners

DataflowPipelineTranslator knows how to translate Pipeline objects into Cloud Dataflow Service API Jobs.

DataflowPipelineTranslator.JobSpecification - Class in com.google.cloud.dataflow.sdk.runners

The result of a job translation.

DataflowPipelineTranslator.TransformTranslator<TransformT extends PTransform> - Interface in com.google.cloud.dataflow.sdk.runners

A DataflowPipelineTranslator.TransformTranslator knows how to translate a particular subclass of PTransform for the Cloud Dataflow service.

DataflowPipelineTranslator.TranslationContext - Interface in com.google.cloud.dataflow.sdk.runners

The interface provided to registered callbacks for interacting with the DataflowPipelineRunner, including reading and writing the values of PCollections and side inputs (PCollectionViews).

DataflowPipelineWorkerPoolOptions - Interface in com.google.cloud.dataflow.sdk.options

Options that are used to configure the Dataflow pipeline worker pool.

DataflowPipelineWorkerPoolOptions.AutoscalingAlgorithmType - Enum in com.google.cloud.dataflow.sdk.options

Type of autoscaling algorithm to use.

DataflowPipelineWorkerPoolOptions.TeardownPolicy - Enum in com.google.cloud.dataflow.sdk.options

Deprecated.

Dataflow Service will only support TEARDOWN_ALWAYS policy in the future.

DataflowPipelineWorkerPoolOptions.WorkerHarnessContainerImageFactory - Class in com.google.cloud.dataflow.sdk.options

Returns the default Docker container image that executes Dataflow worker harness, residing in Google Container Registry.

DataflowProfilingAgentConfiguration() - Constructor for class com.google.cloud.dataflow.sdk.options.DataflowProfilingOptions.DataflowProfilingAgentConfiguration

 

DataflowProfilingOptions - Interface in com.google.cloud.dataflow.sdk.options

Options for controlling profiling of pipeline execution.

DataflowProfilingOptions.DataflowProfilingAgentConfiguration - Class in com.google.cloud.dataflow.sdk.options

Configuration the for profiling agent.

DataflowServiceException - Exception in com.google.cloud.dataflow.sdk.runners

Signals there was an error retrieving information about a job from the Cloud Dataflow Service.

DataflowWorkerHarnessOptions - Interface in com.google.cloud.dataflow.sdk.options

Options that are used exclusively within the Dataflow worker harness.

DataflowWorkerLoggingOptions - Interface in com.google.cloud.dataflow.sdk.options

Options that are used to control logging configuration on the Dataflow worker.

DataflowWorkerLoggingOptions.Level - Enum in com.google.cloud.dataflow.sdk.options

The set of log levels that can be used on the Dataflow worker.

DataflowWorkerLoggingOptions.WorkerLogLevelOverrides - Class in com.google.cloud.dataflow.sdk.options

Defines a log level override for a specific class, package, or name.

DATASTORE_BATCH_UPDATE_LIMIT - Static variable in class com.google.cloud.dataflow.sdk.io.DatastoreIO

Deprecated.

Datastore has a limit of 500 mutations per batch operation, so we flush changes to Datastore every 500 entities.

DatastoreIO - Class in com.google.cloud.dataflow.sdk.io.datastore

DatastoreIO provides an API for reading from and writing to Google Cloud Datastore over different versions of the Cloud Datastore Client libraries.

DatastoreIO - Class in com.google.cloud.dataflow.sdk.io

Deprecated.

replaced by DatastoreIO

DatastoreIO() - Constructor for class com.google.cloud.dataflow.sdk.io.DatastoreIO

Deprecated.

 

DatastoreIO.DatastoreReader - Class in com.google.cloud.dataflow.sdk.io

Deprecated.

A Source.Reader over the records from a query of the datastore.

DatastoreIO.Sink - Class in com.google.cloud.dataflow.sdk.io

Deprecated.

A Sink that writes a PCollection containing Entities to a Datastore kind.

DatastoreIO.Source - Class in com.google.cloud.dataflow.sdk.io

Deprecated.

A Source that reads the result rows of a Datastore query as Entity objects.

DatastoreReader(DatastoreIO.Source, Datastore) - Constructor for class com.google.cloud.dataflow.sdk.io.DatastoreIO.DatastoreReader

Deprecated.

Returns a DatastoreReader with Source and Datastore object set.

DatastoreV1 - Class in com.google.cloud.dataflow.sdk.io.datastore

DatastoreV1 provides an API to Read, Write and Delete PCollections of Google Cloud Datastore version v1 Entity objects.

DatastoreV1.DeleteEntity - Class in com.google.cloud.dataflow.sdk.io.datastore

A PTransform that deletes Entities from Cloud Datastore.

DatastoreV1.DeleteKey - Class in com.google.cloud.dataflow.sdk.io.datastore

A PTransform that deletes Entities associated with the given Keys from Cloud Datastore.

DatastoreV1.Read - Class in com.google.cloud.dataflow.sdk.io.datastore

A PTransform that reads the result rows of a Cloud atastore query as Entity objects.

DatastoreV1.Write - Class in com.google.cloud.dataflow.sdk.io.datastore

A PTransform that writes Entity objects to Cloud Datastore.

days(int) - Static method in class com.google.cloud.dataflow.sdk.transforms.windowing.CalendarWindows

Returns a WindowFn that windows elements into periods measured by days.

decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.AvroCoder

 

decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.BigEndianIntegerCoder

 

decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.BigEndianLongCoder

 

decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.ByteArrayCoder

 

decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.ByteCoder

 

decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.ByteStringCoder

 

decode(InputStream, Coder.Context) - Method in interface com.google.cloud.dataflow.sdk.coders.Coder

Decodes a value of type T from the given input stream in the given context.

decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.DelegateCoder

 

decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.DoubleCoder

 

decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.DurationCoder

 

decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.EntityCoder

Deprecated.

 

decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.InstantCoder

 

decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.IterableLikeCoder

 

decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.JAXBCoder

 

decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.KvCoder

 

decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.MapCoder

 

decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.NullableCoder

 

decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.Proto2Coder

Deprecated.

 

decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.protobuf.ProtoCoder

 

decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.SerializableCoder

 

decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.StringUtf8Coder

 

decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.TableRowJsonCoder

 

decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.TextualIntegerCoder

 

decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.VarIntCoder

 

decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.VarLongCoder

 

decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.coders.VoidCoder

 

decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.join.CoGbkResult.CoGbkResultCoder

 

decode(InputStream, Coder.Context) - Method in class com.google.cloud.dataflow.sdk.transforms.join.