Package com.google.appengine.api.search (2.0.0)

Provides a service for indexing documents and retrieving them using search queries. This is a low-level API that allows users to directly create Documents which can be indexed and retrieved with the help of Index.

A Document is a collection of Fields. Each field is a named and typed value. A document is uniquely identified by its ID and may contain zero or more fields. A field with a given name can have multiple occurrences. Once documents are put into the Index, they can be retrieved via search queries. Typically, a program creates an index. This operation does nothing if the index was already created. Next, a number of documents are inserted into the index. Finally, index is searched and matching documents, or their snippets are returned to the user.


 public List<ScoredDocument> indexAndSearch(
     String query, Document... documents) {
     SearchService searchService = SearchServiceFactory.getSearchService();
     Index index = searchService.getIndex(
         IndexSpec.newBuilder().setIndexName("indexName"));
     for (Document document : documents) {
       PutResponse response = index.put(document);
       assert response.getResults().get(0).getCode().equals(StatusCode.OK);
     }
     Results<ScoredDocument> results =
         index.search(Query.newBuilder().build(query));
     List<ScoredDocument> matched = new ArrayList<ScoredDocument>(
         results.getNumberReturned());
     for (ScoredDocument result : results) {
       matched.add(result);
     }
     return matched;
 }

Classes

AdminSearchServiceFactory

Builds SearchService instances that are pinned to a specific application and namespace regardless of the "current" appId provided by ApiProxy.getCurrentEnvironment().getAppId() and the "current" namespace provided by NamespaceManager.get().

Note: users should not access this class directly.

Cursor

Represents a cursor on the set of results found for executing a Query during a search on the Index.

For example, the following code shows how to use a cursor to get the next page of results


 Index index = ...
 Cursor cursor = Cursor.newBuilder().build();
 Query query = Query.newBuilder().setOptions(
     QueryOptions.newBuilder().setCursor(cursor).build("some query"));

 // Get the first page of results
 Results<ScoredDocument> results = index.search(query);

 // process results
 ...

 // Get the next set of results from the returned cursor
 query = Query.newBuilder().setOptions(
     QueryOptions.newBuilder().setCursor(
         results.getCursor()).build("some query"));

 results = index.search(query);

Alternatively, you can get a cursor to continue from each of the returned results.


 Cursor cursor =
     Cursor.newBuilder().setPerResult(true).build();
 Query query = Query.newBuilder().setOptions(
     QueryOptions.newBuilder().setCursor(cursor).build("some query"));

 // Get the first page of results
 Results<ScoredDocument> results = index.search(query);

 // process results
 for (ScoredDocument result : results) {
   // choose a cursor from one of the results
   cursor = result.getCursor();
 }

 // Get the next set of results from the result's cursor
 query = Query.newBuilder().setOptions(
     QueryOptions.newBuilder().setCursor(cursor).build("some query"));

 results = index.search(query);

Cursor.Builder

A builder which constructs Cursor objects.

DateUtil

A utility class that centralizes processing of dates.

Document

Represents a user generated document. The following example shows how to create a document consisting of a set of fields, some with plain text and some in HTML; it also adds facets to the document.


    Document document = Document.newBuilder().setId("document id")
       .setLocale(Locale.UK)
       .addField(Field.newBuilder()
           .setName("subject")
           .setText("going for dinner"))
       .addField(Field.newBuilder()
           .setName("body")
           .setHTML("I found a restaurant."))
       .addField(Field.newBuilder()
           .setName("signature")
           .setText("ten post jest przeznaczony dla odbiorcy")
           .setLocale(new Locale("pl")))
       .addFacet(Facet.withAtom("tag", "food"))
       .addFacet(Facet.withNumber("priority", 5.0))
       .build();

The following example shows how to access the fields within a document:


    Document document = ...

    for (Field field : document.getFields()) {
      switch (field.getType()) {
        case TEXT: use(field.getText()); break;
        case HTML: use(field.getHtml()); break;
        case ATOM: use(field.getAtom()); break;
        case DATE: use(field.getDate()); break;
      }
    }

And this example shows how to access the facets within a document:


    Document document = ...

    for (Facet facet : document.getFacets()) {
      switch (facet.getType()) {
        case ATOM:   use(facet.getAtom()); break;
        case NUMBER: use(facet.getNumber()); break;
      }
    }

Document.Builder

A builder of documents. This is not thread-safe.

A Facet can be used to categorize a Document. It is not a Field.

Search results can contain facets for the extended result set and their value frequency. For example, if a search query is related to "wine", then facets could be "color" with values of "red" and "white", and "year" with values of "2000" and "2005".

Each facet has a name and exactly one value: atom or number. Facet name lengths are between 1 and com.google.appengine.api.search.checkers.SearchApiLimits#MAXIMUM_NAME_LENGTH characters, and atoms are limited to com.google.appengine.api.search.checkers.SearchApiLimits#MAXIMUM_ATOM_LENGTH characters. Numbers must be between com.google.appengine.api.search.checkers.SearchApiLimits#MINIMUM_NUMBER_VALUE and com.google.appengine.api.search.checkers.SearchApiLimits#MAXIMUM_NUMBER_VALUE.

FacetOptions

A FacetOptions represents facet options such as the number of facets to discover (discoveryLimit), the number of values to be included in each discovered facet (discoveryValueLimit), and the depth of the results to check (depth). Note that discovery is disabled when discoveryLimit is zero.

For example, to discover 10 facets with 5 values each over 1000 extended results:


   FacetOptions facetOption = FacetOptions.newBuilder()
       .setDiscoverLimit(10)
       .setDiscoverValueLimit(5)
       .setDepth(1000)
       .build();

FacetOptions.Builder

Builder for FacetOptions.

FacetRange

A FacetRange is a range with a start (inclusive) and an end (exclusive).

FacetRefinement

A Facet Refinement to filter out search results based on a facet value.

We recommend using refinement token strings instead of this class. We include a refinement token string with each FacetResultValue returned by the backend that can be passed to Query.Builder#addFacetRefinementFromToken(String) to refine follow-up queries.

We also support manually-specified query refinements by passing an instance of this class to Query.Builder#addFacetRefinement(FacetRefinement).

Example: Request to only return documents that have a number facet named "rating" with a value between one and two:


 FacetRefinement lowRating = FacetRefinement.withRange("rating", FacetRange.startEnd(1.0, 2.0));
 query.addFacetRefinement(lowRating);

FacetRequest

A facet request representing parameters for requesting specific facets to be returned with a query result.

For example, to request a facet with a name and specific values:


   FacetRequest request = FacetRequest.newBuilder().setName("wine_type")
       .addValueConstraint("white").addValueConstraint("red").build();

and to request ranges:


 FacetRequest request = FacetRequest.newBuilder().setName("year")
     .addRange(null, 2000.0)           // year < 2000.0="" .addrange(1980.0,="" 2000.0)="" 1980.0=""><= year="">< 2000.0="" .addrange(2000.0,="" null).build();="" year="">= 2000.0

FacetRequest.Builder

A facet request builder. Each facet request should at least have the name of the facet. It can also includes number of values, a list of constraints on the values or a list of ranges for numeric facets. Note that the list of constraints and the list of ranges are mutually exclusive, i.e. you can specify one of them but not both.

FacetResult

Represents a facet result computed from an extended search result set. A facet result contains a name, a type, and a set of values. Name is a single facet name and each value has a label and a count. The value label can be a single facet value name, or a range label (in "[start,end)" format).

FacetResult.Builder

A builder of facet result. This is not thread-safe.

FacetResultValue

Represents a single facet result value. The value has a label, a count, and a refinementToken.

Field

Represents a field of a Document, which is a name, an optional locale, and at most one value: text, HTML, atom, date, GeoPoint, untokenizedPrefix, tokenizedPrefix or vector. Field name lengths are between 1 and com.google.appengine.api.search.checkers.SearchApiLimits#MAXIMUM_NAME_LENGTH characters, and text and HTML values are limited to com.google.appengine.api.search.checkers.SearchApiLimits#MAXIMUM_TEXT_LENGTH. Atoms are limited to com.google.appengine.api.search.checkers.SearchApiLimits#MAXIMUM_ATOM_LENGTH characters, both prefix types are limited to com.google.appengine.api.search.checkers.SearchApiLimits#MAXIMUM_PREFIX_LENGTH . Vector field size is limited to com.google.appengine.api.search.checkers.SearchApiLimits#VECTOR_FIELD_MAX_SIZE and dates must not have a time component.

There are 5 types of text fields, ATOM, TEXT, HTML, UNTOKENIZED_PREFIX, and TOKENIZED_PREFIX. Atom fields when queried, are checked for equality. For example, if you add a field with name code and an ATOM value of "928A 33B-1", then query code:"928A 33B-1" would match the document with this field, while query code:928A would not. TEXT fields, unlike ATOM, match both on equality or if any token extracted from the original field matches. Thus if code field had the value set with Field.Builder#setText(String) method, both queries would match. HTML fields have HTML tags stripped before tokenization. Untokenized prefix fields match queries that are prefixes containing the contiguous starting characters of the whole field. For example if the field was "the quick brown fox", the query "the qui" would match whereas "th qui" would not. On the other hand, Tokenized prefix fields match if the query terms are prefixes of individual terms in the field. If the query is a phrase of terms, the ordering of the terms will matter. For example if the field is "the quick brown fox", the query "th qui bro" would match whereas "bro qui the" would not. Vector fields are only used to compute the dot product between a given constant vector and the provided vector field for sorting and field expressions only. for example, if a 3d vector is named "scores" and has a value of (1,2,3) then the expression dot(scores, vector(3,2,1)) will be evaluated to 10.

Field.Builder

A field builder. Fields must have a name, and optionally a locale and at most one of text, html, atom or date.

FieldExpression

Represents an expression bound to a returned Field with the given name. FieldExpressions are added to a QueryOptions, to request an expression computed and returned as the named Field value. For example,


   FieldExpression.newBuilder()
       .setName("snippet")
       .setExpression("snippet("good story", content)")
       .build()

binds a snippet expression to a Field named "snippet", which will be returned in each search result. In this case the returned "snippet" Field will contain a HTML value containing text snippets of the "content" field matching the query "good story".

FieldExpression.Builder

A field expression builder. A name and expression must be supplied.

GeoPoint

Represents a point on the Earth's surface, in latitude and longitude coordinates.

GetIndexesRequest

A request to get a range of indexes. You can specify a number of restrictions, such as the number of indexes to return, the prefix with which names of the returned indexes must begin, etc.

A namespace may be specified, otherwise the default namespace will be used. Only the indexes defined in the namespace, default or otherwise, will be returned.


   GetIndexesRequest request = GetIndexesRequest.newBuilder()
       .setIndexNamePrefix("a")
       .setOffset(100)
       .setLimit(10)
       .build();

GetIndexesRequest.Builder

The builder of GetIndexesRequests.

GetRequest

A request to list objects in an index. You can specify a number of restrictions, such as the number of objects to return, the id of the first object to return, whether to only return keys, etc.


   GetRequest request = GetRequest.newBuilder()
       .setLimit(500)
       .setStartId("some-id")
       .setReturningIdsOnly(true)
       .build();

GetRequest.Builder

The builder of GetRequests.

GetResponse<T>

Represents a result of executing a GetRequest. The response contains a list of T.

ISearchServiceFactoryProvider

Factory provider for ISearchServiceFactory.

Note: This class is not intended for end users.

IndexSpec

Represents information about an index. This class is used to fully specify the index you want to retrieve from the SearchService. To build an instance use the #newBuilder() method and set all required parameters, plus optional values different than the defaults.


   SearchService searchService = SearchServiceFactory.getSearchService();

   IndexSpec spec = IndexSpec.newBuilder()
       .setName("docs")
       .build();

   Index index = searchService.getIndex(spec);

IndexSpec.Builder

A builder of IndexSpec.

MatchScorer

Assigns a document score based on frequency of terms in TextFields and HTMLFields.

If you add a MatchScorer to a SortOptions as in the following code:


  SortOptions sortOptions = SortOptions.newBuilder()
      .setMatchScorer(MatchScorer.newBuilder())
      .build();

then this will sort the documents in descending score order. The scores will be positive. If you want to sort in ascending order, then use the following code:


   SortOptions sortOptions = SortOptions.newBuilder()
       .setMatchScorer(MatchScorer.newBuilder())
       .addSortExpression(
           SortExpression.newBuilder()
               .setExpression(SortExpression.SCORE_FIELD_NAME)
               .setDirection(SortExpression.SortDirection.ASCENDING)
               .setDefaultValueNumeric(0.0))
       .build();

In this example, the score will be negative.

MatchScorer.Builder

A builder that constructs MatchScorers. A MatchScorer will invoke a scorer on each search result. The following code illustrates building a match scorer to score documents:


   MatchScorer scorer = MatchScorer.newBuilder().build();

OperationResult

The result of an operation involving the search service.

PutResponse

Represents a result of putting a list of objects (documents or queries) into an index. The response contains a list of OperationResult indicating success or not of putting each of the objects into the index, and a list of Id of the objects which are those given in the request or allocated by the search service to those objects which do not have an Id supplied.

Query

A query to search an index for documents which match, restricting the document fields returned to those given, and scoring and sorting the results, whilst supporting pagination.

For example, the following query will search for documents where the tokens 'good' and 'story' occur in some fields, returns up to 20 results including the fields 'author' and 'date-sent' as well as snippeted fields 'subject' and 'body'. The results are sorted by 'author' in descending order, getting the next 20 results from the responseCursor in the previously returned results, giving back a single cursor in the Results to get the next batch of results after this.


 QueryOptions options = QueryOptions.newBuilder()
     .setLimit(20)
     .setFieldsToSnippet("subject", "body")
     .setScorer(CustomScorer.newBuilder()
         .addSortExpression(SortExpression.newBuilder()
             .setExpression("author")
             .setDirection(SortDirection.DESCENDING)
             .setDefaultValue("")))
     .setCursor(responseCursor)
     .build();
 Query query = Query.newBuilder()
     .setOptions(options)
     .build("good story");

The following query will return facet information with the query result:

Query query = Query.newBuilder() .setOptions(options) .setEnableFacetDiscovery(true) .build("tablet");

To customize returned facet or refine the result using a previously returned FacetResultValue#getRefinementToken:

Query query = Query.newBuilder() .setOptions(options) .setEnableFacetDiscovery(true) .setFacetOptions(FacetOptions.newBuilder().setDiscoveryLimit(5).build()) .addReturnFacet("shipping") .addReturnFacet(FacetRequest.newBuilder().setName("department") .addValueConstraint("Computers") .addValueConstraint("Electronics") .build()) .addRefinementToken(refinementToken1) .addRefinementToken(refinementToken2) .build("tablet");

Query.Builder

A builder which constructs Query objects.

QueryOptions

Represents options which control where and what in the search results to return, from restricting the document fields returned to those given, and scoring and sorting the results, whilst supporting pagination.

For example, the following options will return documents from search results for some given query, returning up to 20 results including the fields 'author' and 'date-sent' as well as snippeted fields 'subject' and 'body'. The results are sorted by 'author' in descending order, getting the next 20 results from the responseCursor in the previously returned results, giving back a single cursor in the Results to get the next batch of results after this.


 QueryOptions request = QueryOptions.newBuilder()
      .setLimit(20)
      .setFieldsToReturn("author", "date-sent")
      .setFieldsToSnippet("subject", "body")
      .setSortOptions(SortOptions.newBuilder().
          .addSortExpression(SortExpression.newBuilder()
              .setExpression("author")
              .setDirection(SortExpression.SortDirection.DESCENDING)
              .setDefaultValue("")))

      .setCursor(Cursor.newBuilder().build())
      .build();

QueryOptions.Builder

A builder which constructs QueryOptions objects.

RequestStatusUtil

Collection of utility methods for SearchServicePb.RequestStatus.

RescoringMatchScorer

Assigns a document score based on term frequency weighted on document parts.

If you add a MatchScorer to a SortOptions as in the following code:


  SortOptions sortOptions = SortOptions.newBuilder()
      .setMatchScorer(RescoringMatchScorer.newBuilder())
      .build();

then this will sort the documents in descending score order. The scores will be positive. If you want to sort in ascending order, then use the following code:


   SortOptions sortOptions = SortOptions.newBuilder()
       .setMatchScorer(RescoringMatchScorer.newBuilder())
       .addSortExpression(
           SortExpression.newBuilder()
               .setExpression(SortExpression.SCORE_FIELD_NAME)
               .setDirection(SortExpression.SortDirection.ASCENDING)
               .setDefaultValueNumeric(0.0))
       .build();

The scores in this case will be negative.

RescoringMatchScorer.Builder

A builder that constructs RescoringMatchScorers. A RescoringMatchScorer will invoke a scorer on each search result. The following code illustrates building a rescoring match scorer to score documents:

RescoringMatchScorer scorer = RescoringMatchScorer.newBuilder().build();

Results<T>

Represents a result of executing a search. The Results include an OperationResult, a collection of results, and a number of found and returned results.

Schema

Contains information about the kinds of document Fields which are supported by the Index.


   // Get the searchService for the default namespace
   SearchService searchService = SearchServiceFactory.getSearchService();

   // Get the first page of indexes available and retrieve schemas
   GetResponse<Index> response = searchService.getIndexes(
       GetIndexesRequest.newBuilder().setSchemaFetched(true).build());

   // List out elements of Schema
   for (Index index : response) {
     Schema schema = index.getSchema();
     for (String fieldName : schema.getFieldNames()) {
        List<FieldType> typesForField = schema.getFieldTypes(fieldName);
     }
   }

Schema.Builder

A builder which constructs Schema objects.

ScoredDocument

Represents a document which may have been scored, possibly some computed expression fields, and a cursor to continue the search from.

ScoredDocument.Builder

A builder of scored documents. This is not thread-safe.

SearchServiceConfig

Configuration options for Search API.

SearchServiceConfig.Builder

Builder for SearchServiceConfig.

SearchServiceFactory

An factory that creates default implementation of SearchService.


   SearchService search = SearchServiceFactory.getSearchService();

Optionally, you may pass a SearchServiceConfig instance to customize the search service. e.g, setting deadline and namespace:


   SearchServiceFactory.getSearchService(
       SearchServiceConfig.newBuilder().setDeadline(10.0).setNamespace("acme").build())

SortExpression

Sorting specification for a single dimension. Multi-dimensional sorting is supported by a collection of SortExpressions.

SortExpression.Builder

A builder that constructs SortExpressions. The user must provide an expression. The expression can be as simple as a field name, or can be some other expression such as "score + count(likes) * 0.1", which combines a scorer score with a count of the number of likes values times 0.1. A default value must be specified for the expression.


   SortExpression expr = SortExpression.newBuilder()
       .setExpression(String.format(
           "%s + count(likes) * 0.1", SortExpression.SCORE_FIELD_NAME))
       .setDirection(SortExpression.SortDirection.ASCENDING)
       .setDefaultValueNumeric(0.0)
       .build()

SortOptions

Definition of how to sort documents. You may specify zero or more sort expressions and set a match scorer. If you have a large index, it is advisable to set a limit.

SortOptions.Builder

A builder that constructs SortOptionss. A SortOptions will evaluate each of the SortExpressions on each search result and apply a sort order with priority given to the sort expressions from left to right. The following code illustrates creating a SortOptions specification to sort documents based on decreasing product rating and then within rating showing cheapest products based on price plus tax, sorting at most 2000 documents.


   SortOptions sortOptions = SortOptions.newBuilder()
       .addSortExpression(SortExpression.newBuilder()
           .setExpression("rating")
           .setDirection(SortExpression.SortDirection.DESCENDING)
           .setDefaultValueNumeric(0))
       .addSortExpression(SortExpression.newBuilder()
           .setExpression("price + tax")
           .setDirection(SortExpression.SortDirection.ASCENDING)
           .setDefaultValueNumeric(99999999.00))
       .setLimit(1000)
       .build();

The following code fragment shows how the score from a MatchScorer can be used in an expression that combines the score with one thousandth of an "importance" field. At most 1000 documents are scored and sorted.


   SortOptions sortOptions = SortOptions.newBuilder()
       .setMatchScorer(MatchScorer.newBuilder())
       .addSortExpression(SortExpression.newBuilder()
           .setExpression(String.format(
               "%s + (importance * .001)", SortExpression.SCORE_FIELD_NAME))
           .setDirection(SortExpression.SortDirection.DESCENDING)
           .setDefaultValueNumeric(0))
       .setLimit(1000)
       .build();

Interfaces

ISearchServiceFactory

A factory that creates default implementation of SearchService.

Index

An Index allows synchronous and asynchronous adding and deleting of Documents as well as synchronous and asynchronous searching for Documents for a given Query. The following code fragment shows how to add documents, then search the index for documents matching a query.


  // Get the SearchService for the default namespace
  SearchService searchService = SearchServiceFactory.getSearchService();
  // Get the index. If not yet created, create it.
  Index index = searchService.getIndex(
      IndexSpec.newBuilder().setIndexName("indexName"));

  // Create a document.
  Document document = Document.newBuilder()
      .setId("documentId")
      .addField(Field.newBuilder().setName("subject").setText("my first email"))
      .addField(Field.newBuilder().setName("body")
           .setHTML(some content here")
      .build();

  // Put the document.
  try {
    index.put(document);
  } catch (PutException e) {
    if (StatusCode.TRANSIENT_ERROR.equals(e.getOperationResult().getCode())) {
      // retry putting document
    }
  }

  // Query the index.
  try {
    Results<ScoredDocument> results =
        index.search(Query.newBuilder().build("subject:first body:here"));

    // Iterate through the search results.
    for (ScoredDocument document : results) {
      // display results
    }
  } catch (SearchException e) {
    if (StatusCode.TRANSIENT_ERROR.equals(e.getOperationResult().getCode())) {
      // retry
    }
  }

SearchService

The SearchService is used to get available indexes, which can be queried about their metadata or have index/delete/search operations performed on them. For example:


 SearchService searchService = SearchServiceFactory.getSearchService();
 GetResponse<Index> response = searchService.getIndexes(
     GetIndexesRequest.newBuilder());
 for (Index index : response) {
   index.getName();
   index.getNamespace();
   index.search("query");
 }

SearchService is also responsible for creating new indexes. For example:


 SearchService searchService = SearchServiceFactory.getSearchService();
 Index index = searchService.getIndex(IndexSpec.newBuilder().setName("myindex"));

Package com.google.appengine.api.search (2.0.0)

Classes

Interfaces

Enums

Exceptions