Google Cloud Dataflow SDK for Java, version 1.9.1
Class DatastoreV1.Read
- java.lang.Object
-
- com.google.cloud.dataflow.sdk.transforms.PTransform<PBegin,PCollection<com.google.datastore.v1.Entity>>
-
- com.google.cloud.dataflow.sdk.io.datastore.DatastoreV1.Read
-
- All Implemented Interfaces:
- HasDisplayData, Serializable
- Enclosing class:
- DatastoreV1
public static class DatastoreV1.Read extends PTransform<PBegin,PCollection<com.google.datastore.v1.Entity>>
APTransform
that reads the result rows of a Cloud atastore query asEntity
objects.- See Also:
DatastoreIO
, Serialized Form
-
-
Field Summary
Fields Modifier and Type Field and Description static int
NUM_QUERY_SPLITS_MAX
An upper bound on the number of splits for a query.-
Fields inherited from class com.google.cloud.dataflow.sdk.transforms.PTransform
name
-
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method and Description PCollection<com.google.datastore.v1.Entity>
apply(PBegin input)
Applies thisPTransform
on the givenInputT
, and returns itsOutput
.String
getNamespace()
String
getProjectId()
com.google.datastore.v1.Query
getQuery()
void
populateDisplayData(DisplayData.Builder builder)
Register display data for the given transform or component.String
toString()
void
validate(PBegin input)
Called before invoking apply (which may be intercepted by the runner) to verify this transform is fully specified and applicable to the specified input.DatastoreV1.Read
withNamespace(String namespace)
Returns a newDatastoreV1.Read
that reads from the given namespace.DatastoreV1.Read
withNumQuerySplits(int numQuerySplits)
DatastoreV1.Read
withProjectId(String projectId)
Returns a newDatastoreV1.Read
that reads from the Cloud Datastore for the specified project.DatastoreV1.Read
withQuery(com.google.datastore.v1.Query query)
Returns a newDatastoreV1.Read
that reads the results of the specified query.-
Methods inherited from class com.google.cloud.dataflow.sdk.transforms.PTransform
getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName
-
-
-
-
Field Detail
-
NUM_QUERY_SPLITS_MAX
public static final int NUM_QUERY_SPLITS_MAX
An upper bound on the number of splits for a query.- See Also:
- Constant Field Values
-
-
Method Detail
-
withProjectId
public DatastoreV1.Read withProjectId(String projectId)
Returns a newDatastoreV1.Read
that reads from the Cloud Datastore for the specified project.
-
withQuery
public DatastoreV1.Read withQuery(com.google.datastore.v1.Query query)
Returns a newDatastoreV1.Read
that reads the results of the specified query.Note: Normally,
DatastoreIO
will read from Cloud Datastore in parallel across many workers. However, when theQuery
is configured with a limit usingQuery.Builder#setLimit
, then all results will be read by a single worker in order to ensure correct results.
-
withNamespace
public DatastoreV1.Read withNamespace(String namespace)
Returns a newDatastoreV1.Read
that reads from the given namespace.
-
withNumQuerySplits
public DatastoreV1.Read withNumQuerySplits(int numQuerySplits)
Returns a newDatastoreV1.Read
that reads by splitting the givenquery
intonumQuerySplits
.The semantics for the query splitting is defined below:
- Any value less than or equal to 0 will be ignored, and the number of splits will be chosen dynamically at runtime based on the query data size.
- Any value greater than
NUM_QUERY_SPLITS_MAX
will be capped atNUM_QUERY_SPLITS_MAX
. - If the
query
has a user limit set, thennumQuerySplits
will be ignored and no split will be performed. - Under certain cases Cloud Datastore is unable to split query to the requested number of splits. In such cases we just use whatever the Datastore returns.
-
getQuery
@Nullable public com.google.datastore.v1.Query getQuery()
-
getProjectId
@Nullable public String getProjectId()
-
getNamespace
@Nullable public String getNamespace()
-
apply
public PCollection<com.google.datastore.v1.Entity> apply(PBegin input)
Applies thisPTransform
on the givenInputT
, and returns itsOutput
.Composite transforms, which are defined in terms of other transforms, should return the output of one of the composed transforms. Non-composite transforms, which do not apply any transforms internally, should return a new unbound output and register evaluators (via backend-specific registration methods).
The default implementation throws an exception. A derived class must either implement apply, or else each runner must supply a custom implementation via
PipelineRunner.apply(com.google.cloud.dataflow.sdk.transforms.PTransform<InputT, OutputT>, InputT)
.- Overrides:
apply
in classPTransform<PBegin,PCollection<com.google.datastore.v1.Entity>>
-
validate
public void validate(PBegin input)
Description copied from class:PTransform
Called before invoking apply (which may be intercepted by the runner) to verify this transform is fully specified and applicable to the specified input.By default, does nothing.
- Overrides:
validate
in classPTransform<PBegin,PCollection<com.google.datastore.v1.Entity>>
-
populateDisplayData
public void populateDisplayData(DisplayData.Builder builder)
Description copied from class:PTransform
Register display data for the given transform or component.populateDisplayData(DisplayData.Builder)
is invoked by Pipeline runners to collect display data viaDisplayData.from(HasDisplayData)
. Implementations may callsuper.populateDisplayData(builder)
in order to register display data in the current namespace, but should otherwise usesubcomponent.populateDisplayData(builder)
to use the namespace of the subcomponent.By default, does not register any display data. Implementors may override this method to provide their own display data.
- Specified by:
populateDisplayData
in interfaceHasDisplayData
- Overrides:
populateDisplayData
in classPTransform<PBegin,PCollection<com.google.datastore.v1.Entity>>
- Parameters:
builder
- The builder to populate with display data.- See Also:
HasDisplayData
-
toString
public String toString()
- Overrides:
toString
in classPTransform<PBegin,PCollection<com.google.datastore.v1.Entity>>
-
-