This page introduces search with answer and follow-ups for Vertex AI Search and shows you how to implement it for website data and unstructured data using method calls.
Search with answer and follow-ups is based on the answer method. The answer method encompasses some of the features of the current search and converse methods. The answer method also has some important additional features, such as the ability to handle complex queries.
About the searching methods
Vertex AI Search has three methods that can return search results for apps:
Search. The search method is for returning search results from data stores and websites.
For information about the search method and how to use it, see Search and Get search results.
Conversation. The converse method is for searching with follow-up questions.
For information about the converse method and how to use it, see Converse and Search with follow-ups.
Answer. This method is in Preview. The answer method has many features in common with the search method and all the features of the converse method.
For information about the answer method, see Answer in the API reference and this guide.
To decide which method you should use, see When not to use the answer method.
Key features of the answer method
The key advantages of the answer method over the search and the converse methods are as follows:
The ability to provide answers to complex queries. For example, the answer method can break down compound queries, such as the following, into multiple, smaller queries to return better results that are used to power better answers:
- "What is Google Cloud and Google Ads respective revenue in 2024?"
- "After how many years since its founding, did Google reach 1 billion USD revenue?"
The ability to combine search and answer generation in a multi-turn conversation by calling the answer API in each turn. The answer method has all the features of the converse method and some of the features of the search method.
The ability to reduce search latency. You can call the search method and the answer method separately and render the search results and answers in different iframes at different times. This means that you can show your users search results (the 10 blue links) within milliseconds. You don't need to wait for answers to be generated before you can show search results.
The features of answer and follow-ups can be divided into three phases of the query, search and answer:
Query phase features
The answer and follow-ups feature supports natural language query processing.
This section describes and illustrates the various options for query rephrasing and classification.
Query rephrasing
Query rephrasing is on by default. This feature chooses the best way to rephrase queries automatically to improve search results. This feature can also handle queries that don't require rephrasing.
Break down complex queries into multiple queries and perform synchronous sub-querying.
For example: A complex query is broken down into four smaller, simpler queries.
User input Sub-queries created from the complex query What jobs and hobbies do Andie Ram and Arnaud Clément have in common? - Andie Ram occupation
- Arnaud Clément occupation
- Andie Ram hobby
- Arnaud Clément hobby
Synthesize multi-turn queries, to make follow-up questions context aware and stateful.
For example: Queries synthesized from user input at each turn might look like this:
User input Query synthesized Turn 1: laptops for school laptops for school Turn 2: not mac laptops for school not mac Turn 3: bigger screen and i also need wireless keyboard and mouse bigger screen laptops for school not mac with wireless keyboard and mouse Turn 4: and a backpack for it bigger screen laptops for school not mac with wireless keyboard and mouse and a backpack for it Simplify long queries to improve retrieval.
For example: A long query is shortened to a simple query.
User input Query simplified I am trying to find out why the \"Add to Cart\" button on our website is not working properly. It seems that when a user clicks the button, the item is not added to the cart and they receive an error message. I have checked the code and it seems to be correct, so I am not sure what the issue could be. Can you help me troubleshoot this problem? "Add to Cart" button not working on website. Perform multi-step reasoning.
Multi-step reasoning is based on the ReAct (reason + act) paradigm which enables LLMs to solve complex tasks using natural language reasoning. By default, the maximum number of steps is 10.
For example:
User input Two steps to generate the answer After how many years since its founding, did Google reach 1 billion USD revenue? Step 1:
[Thought]: I need to know when Google was founded, then I can query it is revenue since then.
[Act] Search: When was Google founded?[Observe Search Results]: "1998"
Step 2:
[Thought]: Now I need to Google's yearly revenue since 1998, and find out when it exceeded 1 billion for the first time.
[Act] Search: Google revenue since 1998
[Observe Search Results] Google revenue in 1998, Google revenue in 1999…..
[Answer]: Google reached more than 1 billion USD revenue in 2003 [1], 5 years after its founding in 1998[2].
Query classification
Query classification options are to identify adversarial queries and non-answer seeking queries. By default, query classification options are off.
For more information about adversarial and non-answer seeking queries, see Ignore adversarial queries and Ignore non-summary seeking queries.
Search phase features
For searching, the answer method has the same options as the search method. For example:
Applying filters to restrict search to specific documents. For more information, see Filter generic search for structured or unstructured data.
Applying SafeSearch to filter out explicit content, such as violence and pornography. For more information, see Safety settings for Vertex AI Search.
Specifying boost conditions to promote or demote documents returned by search. For more information, see Boost search results.
Answer phase features
During the answer phase, when answers are generated from the search results, you can enable the same features as in the search method. For example:
Getting citations to indicate a source for each sentence in the answer. For more information, see Get citations.
Use the prompt preamble to customize the answer for such things as tone, style, and verbosity. For more information, see Specify a custom preamble.
Choose which Vertex AI model to use for answer generation. For more information, see Answer generation model versions and lifecycle.
Choose whether to ignore queries that have been classified as adversarial or non-answer seeking.
For more information about adversarial and non-answer seeking queries, see Ignore adversarial queries and Ignore non-summary seeking queries. Non-answer seeking queries are also called non-summary seeking queries.
When not to use the answer method
In most cases, you'll want to use the answer method instead of the search or converse methods. However, there are a few situations where you must use another method. These are given in the following table.
| Conditions | Use this method instead |
|---|---|
| You only need search results (not answers) | search |
| You need to show more than 10 search results | search |
| You want research result latency within milliseconds | search |
| You have structured data | search |
| You need production (GA) quality | search or converse |
| You want to configure search using the UI panel for a widget or web app | search or converse |
| You want to query multiple data stores | search |
| You want to bring your own embeddings | search |
| You want to apply synonym or redirect controls, facets, or user country codes to your search | search |
If you are new to Vertex AI Search, Google recommends that you use the answer method instead of the converse method.
Before you begin
Depending on the type of app you have, complete the following requirements:
If you have an unstructured search app, make sure the following is turned on: Advanced LLM features
If you have a website search app, make sure the following are turned on:
Search and answer (basic)
The following command shows how to call the answer method and
return a generated answer and a list of search results, with links to the
sources.
This command shows the required input only. The options are left at their defaults.
REST
To search and get results with generative answer, do the following:
Run the following curl command:
curl -X POST -H "Authorization: Bearer $(gcloud auth print-access-token)" \ -H "Content-Type: application/json" \ "https://discoveryengine.googleapis.com/v1beta/projects/PROJECT_ID/locations/global/collections/default_collection/dataStores/DATA_STORE_ID/servingConfigs/default_search:answer" \ -d '{ "query": { "text": "QUERY"} }'Replace the following:
PROJECT_ID: the ID of your Google Cloud project.DATA_STORE_ID: the ID of the Vertex AI Search data store.QUERY: a free-text string that contains the question or search query. For example, "Compare the BigQuery and Spanner databases?".
Query phase commands
This section shows how to specify options for the query phase of the
answer method call.
Search and answer (rephrasing disabled)
The following command shows how to call the answer method and
return a generated answer and a list of search results. The answer could be
different from the preceding answer because the rephrasing option is disabled.
REST
To search and get results with generative answer without applying query rephrasing, do the following:
Run the following curl command:
curl -X POST -H "Authorization: Bearer $(gcloud auth print-access-token)" \ -H "Content-Type: application/json" \ "https://discoveryengine.googleapis.com/v1beta/projects/PROJECT_ID/locations/global/collections/default_collection/dataStores/DATA_STORE_ID/servingConfigs/default_search:answer" \ -d '{ "query": { "text": "QUERY"}, "query_understanding_spec": { "query_rephraser_spec": { "disable": true }'Replace the following:
PROJECT_ID: the ID of your Google Cloud project.DATA_STORE_ID: the ID of the Vertex AI Search data store.QUERY: a free-text string that contains the question or search query. For example, "Compare the BigQuery and Spanner databases?".
Search and answer (specify maximum steps)
The following command shows how to call the answer method and
return a generated answer and a list of search results. The answer is
different from the preceding answers because the number of rephrasing steps has
been increased.
REST
To search and get results with generative answer allowing up to 10 rephrasing steps, do the following:
Run the following curl command:
curl -X POST -H "Authorization: Bearer $(gcloud auth print-access-token)" \ -H "Content-Type: application/json" \ "https://discoveryengine.googleapis.com/v1beta/projects/PROJECT_ID/locations/global/collections/default_collection/dataStores/DATA_STORE_ID/servingConfigs/default_search:answer" \ -d '{ "query": { "text": "QUERY"}, "query_understanding_spec": { "query_rephraser_spec": { "max_rephrase_steps": MAX_REPHRASE }'Replace the following:
PROJECT_ID: the ID of your Google Cloud project.DATA_STORE_ID: the ID of the Vertex AI Search data store.QUERY: a free-text string that contains the question or search query. For example, "Compare the BigQuery and Spanner databases?".MAX_REPHRASE: the maximum number of rephrase steps. The largest value allowed is5. If not set or if set to less than1the value is the default,1
Search and answer with query classification
The following command shows how to call the answer method to
inquire whether a query is adversarial, non-answer seeking, or neither.
The response includes the classification type for the query, but the answer itself is not affected by the classification. If you want to change the answer behavior according to the query type, you can do this in the answer phase. See Ignore adversarial queries and Ignore non-summary seeking queries.
REST
To determine if a query is adversarial or non-answer seeking, do the following:
Run the following curl command:
curl -X POST -H "Authorization: Bearer $(gcloud auth print-access-token)" \ -H "Content-Type: application/json" \ "https://discoveryengine.googleapis.com/v1beta/projects/PROJECT_ID/locations/global/collections/default_collection/dataStores/DATA_STORE_ID/servingConfigs/default_search:answer" \ -d '{ "query": { "text": "QUERY"}, "query_understanding_spec": { "query_classification_spec": { "types": [QUERY_CLASSIFICATION_TYPE] }'Replace the following:
PROJECT_ID: the ID of your Google Cloud project.DATA_STORE_ID: the ID of the Vertex AI Search data store.QUERY: a free-text string that contains the question or search query. For example, "hello".QUERY_CLASSIFICATION_TYPE: the query types that you want to identify:ADVERSARIAL_QUERY,NON_ANSWER_SEEKING_QUERY, or both.
Search phase commands: Search and answer with search result options
This section shows how to specify options for the search phase portion of the
answer method call, options such as setting the maximum number
of documents returned, boosting, and filtering, and how to get an answer when
you supply your own search results.
The following command shows how to call the answer method and
specify various options for how the search result is returned. (The search
results are independent of the answer.)
REST
To set various options related to which and how search results are returned, do the following:
Run the following curl command:
curl -X POST -H "Authorization: Bearer $(gcloud auth print-access-token)" \ -H "Content-Type: application/json" \ "https://discoveryengine.googleapis.com/v1beta/projects/PROJECT_ID/locations/global/collections/default_collection/dataStores/DATA_STORE_ID/servingConfigs/default_search:answer" \ -d '{ "query": { "text": "QUERY"}, "search_spec": { "search_params": { "max_return_results": MAX_RETURN_RESULTS, "filter": "FILTER", "boost_spec": BOOST_SPEC, "order_by": "ORDER_BY", "search_result_mode": SEARCH_RESULT_MODE } }'Replace the following:
PROJECT_ID: the ID of your Google Cloud project.DATA_STORE_ID: the ID of the Vertex AI Search data store.QUERY: a free-text string that contains the question or search query. For example, "Compare the BigQuery and Spanner databases?"MAX_RETURN_RESULTS: the number of search results to return. The default value is 10.FILTER: the filter specifies which documents are queried. If a document's metadata meets the filter specification, then the document will be queried. For more information, including filter syntax, see Filter generic search for structured or unstructured data.BOOST_SPEC: the boost specification lets you boost certain documents in search results, which can affect the answer. For more information, including the syntax for the boost specification, see Boost search results.ORDER_BY: the order in which documents are returned. Documents can be ordered by a field in a Document object. TheorderByexpression is case-sensitive. If this field is unrecognizable, anINVALID_ARGUMENTis returned.SEARCH_RESULT_MODE: It specifies the search result mode. For more information, see Parse and chunk documents and ContentSearchSpec. This field is only available in the v1alpha version of the API.
Answer phase commands
This section shows how to specify answer-specific options for the
answer method call.
Ignore adversarial queries and non-answer-seeking queries
The following command shows how to avoid answering adversarial queries and
non-answer-seeking queries when calling the answer method.
REST
To skip answering queries that are adversarial or non-answer-seeking, do the following:
Run the following curl command:
curl -X POST -H "Authorization: Bearer $(gcloud auth print-access-token)" \ -H "Content-Type: application/json" \ "https://discoveryengine.googleapis.com/v1beta/projects/PROJECT_ID/locations/global/collections/default_collection/dataStores/DATA_STORE_ID/servingConfigs/default_search:answer" \ -d '{ "query": { "text": "QUERY"}, "answer_generation_spec": { "ignore_adversarial_query": true "ignore_non_answer_seeking_query": true } }'Replace the following:
PROJECT_ID: the ID of your Google Cloud project.DATA_STORE_ID: the ID of the Vertex AI Search data store.QUERY: a free-text string that contains the question or search query.
Specify the answer model
The following command shows how to change the model version used to generate answers.
For information about the supported models, see Answer generation model versions and lifecycle.
REST
To generate an answer using a model different from the default model, do the following:
Run the following curl command:
curl -X POST -H "Authorization: Bearer $(gcloud auth print-access-token)" \ -H "Content-Type: application/json" \ "https://discoveryengine.googleapis.com/v1beta/projects/PROJECT_ID/locations/global/collections/default_collection/dataStores/DATA_STORE_ID/servingConfigs/default_search:answer" \ -d '{ "query": { "text": "QUERY"}, "answer_generation_spec": { "model_spec": { "model_version": "MODEL_VERSION", } }'Replace the following:
PROJECT_ID: the ID of your Google Cloud project.DATA_STORE_ID: the ID of the Vertex AI Search data store.QUERY: a free-text string that contains the question or search query.MODEL_VERSION: the model version that you want to use to generate the answer. For more information, see Answer generation model versions and lifecycle.
Specify a custom preamble
The following command shows how to set a preamble for the generated answer. A preamble contains natural language instructions for customizing the answer. You can request customizations such as length, level of detail, style of output (such as "simple"), language of output, focus of answer, and format (such as tables, bullets, and XML). For example, a preamble might be "Explain like you are a ten years old kid."
REST
To generate an answer using a model different from the default model, do the following:
Run the following curl command:
curl -X POST -H "Authorization: Bearer $(gcloud auth print-access-token)" \ -H "Content-Type: application/json" \ "https://discoveryengine.googleapis.com/v1beta/projects/PROJECT_ID/locations/global/collections/default_collection/dataStores/DATA_STORE_ID/servingConfigs/default_search:answer" \ -d '{ "query": { "text": "QUERY"}, "answer_generation_spec": { "prompt_spec": { "preamble": "PREAMBLE", } } }'Replace the following:
PROJECT_ID: the ID of your Google Cloud project.DATA_STORE_ID: the ID of the Vertex AI Search data store.QUERY: a free-text string that contains the question or search query.PREAMBLE: a natural-language instruction for customizing the answer. For example, tryshow the answer format in an ordered listorgive a very detailed answer.
Include citations
The following command shows how to request citations to be included with the answer.
REST
To generate an answer using a model different from the default model, do the following:
Run the following curl command:
curl -X POST -H "Authorization: Bearer $(gcloud auth print-access-token)" \ -H "Content-Type: application/json" \ "https://discoveryengine.googleapis.com/v1beta/projects/PROJECT_ID/locations/global/collections/default_collection/dataStores/DATA_STORE_ID/servingConfigs/default_search:answer" \ -d '{ "query": { "text": "QUERY"}, "answer_generation_spec": { "include_citations": INCLUDE_CITATIONS } } }'Replace the following:
PROJECT_ID: the ID of your Google Cloud project.DATA_STORE_ID: the ID of the Vertex AI Search data store.QUERY: a free-text string that contains the question or search query.INCLUDE_CITATIONS: specifies whether to include citation metadata in the answer. The default value isfalse.
Set the answer language code
This feature is experimental. If you're interested in using this feature, contact your Google account team.
The following command shows how to set the language code for answers.
REST
To generate an answer using a model different from the default model, do the following:
Run the following curl command:
curl -X POST -H "Authorization: Bearer $(gcloud auth print-access-token)" \ -H "Content-Type: application/json" \ "https://discoveryengine.googleapis.com/v1beta/projects/PROJECT_ID/locations/global/collections/default_collection/dataStores/DATA_STORE_ID/servingConfigs/default_search:answer" \ -d '{ "query": { "text": "QUERY"}, "answer_generation_spec": { "answer_language_code": ANSWER_LANGUAGE_CODE } }'Replace the following:
PROJECT_ID: the ID of your Google Cloud project.DATA_STORE_ID: the ID of the Vertex AI Search data store.QUERY: a free-text string that contains the question or search query.ANSWER_LANGUAGE_CODE: a language code for the answer. For a list of allowed codes, see Tags for Identifying Languages.
Commands for follow-up questions
Follow-ups are multi-turn queries. After the first query in a follow-up session, subsequent "turns" take into account prior interactions. With follow-ups, the answer method can also suggest related questions, which your users can choose instead of entering their own follow-up questions.
All the answer and follow-ups features described in the preceding sections, such as citations, filters, SafeSearch, ignoring certain types of queries, and using a preamble to customize answers can be applied along with follow-ups.
Example of a follow-up session
The following is an example of a session with follow-ups. Suppose that you want to know about vacationing in Mexico:
Turn 1:
You: When is the best time of the year to vacation in Mexico?
Answer with follow-ups: The best time to vacation in Mexico is during the dry season, which runs from November to April.
Turn 2:
You: What is the exchange rate?
Answer with follow-ups: 1 USD is equal to approximately 17.65 Mexican pesos.
Turn 3:
You: What's the average temperature in December?
Answer with follow-ups: The average temperature varies from 70-78°F. Cancun's average is ~ 77°F.
Without follow-ups, your question "What is the exchange rate?" wouldn't be answerable because regular search wouldn't know that you wanted the Mexican exchange rate. Similarly, without follow-ups, there wouldn't be the context needed to give you temperatures specifially for Mexico.
Examples of related questions
When you ask "What is the best time of the year to vacation in Mexico?", in addition to answering your question, answer and follow-ups can suggest other questions that you might ask, such as "What is the cheapest month to vacation in Mexico?" and "What are the tourist months in Mexico?".
After the related questions feature is enabled, questions are returned as strings in the ConverseConversationResponse.
About sessions
To understand how follow-ups work in Vertex AI Search, you need to understand about sessions.
A session is made up of text queries provided by a user and responses provided by Vertex AI Search.
These query and response pairs are sometimes referred to as turns. In the preceding example, the second turn is made up of "What is the exchange rate?" and "1 USD is equal to approximately 17.65 Mexican pesos."
Sessions are stored in the same data store where the unstructured data is kept. In the data store, a session is represented by the session resource. In addition to containing the query and response messages, the session resource has:
A unique name (the session ID).
A state (in-progress or completed).
A user pseudo ID, which is a visitor ID that tracks the user. It can be assigned programmatically.
A start time and an end time.
A turn, which is a query answer pair.
Store session information and get responses
You can use the command line to generate search responses and answers and to store these, along with each query in a session.
REST
To use the command line to create a session and generate responses from the user's input, follow these steps:
Specify the data store where you want to store the session:
curl -X POST \ -H "Authorization: Bearer $(gcloud auth print-access-token)" \ -H "Content-Type: application/json" \ "https://discoveryengine.googleapis.com/v1/projects/PROJECT_ID/locations/global/collections/default_collection/dataStores/DATA_STORE_ID/sessions" \ -d '{ "user_pseudo_id": "USER_PSEUDO_ID" }'Replace the following:
PROJECT_ID: the ID of your Google Cloud project.
DATA_STORE_ID: the ID of the Vertex AI Search data store.
USER_PSEUDO_ID: This is a unique identifier for tracking a search visitor. For example, you can implement this with an HTTP cookie, which uniquely identifies a visitor on a single device.
Example command and result
curl -X POST -H "Authorization: Bearer $(gcloud auth print-access-token)" \ -H "Content-Type: application/json" \ "https://discoveryengine.googleapis.com/v1beta/projects/ucs-cs-test/locations/global/collections/default_collection/dataStores/my-data-store/sessions" \ -d '{ "user_pseudo_id": "test_user" }'
{ "name": "projects/7654321/locations/global/collections/default_collection/dataStores/my-data-store/sessions/5386462384953257772", "state": "IN_PROGRESS", "userPseudoId": "test_user", "startTime": "2024-04-03T05:31:30.217535Z", "endTime": "1970-01-01T00:00:00Z" }Generate an answer and add it to a new or existing session in your data store:
curl -X POST \ -H "Authorization: Bearer $(gcloud auth print-access-token)" \ -H "Content-Type: application/json" \ "https://discoveryengine.googleapis.com/v1beta/projects/<var>PROJECT_ID</var>/locations/global/collections/default_collection/dataStores/<var>DATA_STORE_ID</var>/servingConfigs/default_search:answer" \ -d '{ "query": { "text": "<var>QUERY</var>"}, "session": "<var>SESSION_ID</var>", "search_spec":{ "search_params": {"filter": "<var>FILTER</var>"} } }'PROJECT_ID: the ID of your Google Cloud project.DATA_STORE_ID: the ID of the Vertex AI Search data store.QUERY: a free-text string that contains the question or search query.SESSION_ID: A unique ID for the session, for example,123456. For a session, use the same session ID in every turn.FILTER: a text field for filtering search using a filter expression. The default value is an empty string. The way you construct your filter varies depending on whether you have website data or unstructured data with metadata. For more information, see Filter generic search for structured or unstructured data".
Example command and result
curl -X POST -H "Authorization: Bearer $(gcloud auth print-access-token)" -H "Content-Type: application/json" "https://discoveryengine.googleapis.com/v1beta/projects/ucs-cs-test/locations/global/collections/default_collection/dataStores/my-data-store/servingConfigs/default_search:answer" -d '{ "query": { "text": "Compare bigquery with spanner database?"}, "session": "projects/7654321/locations/global/collections/default_collection/dataStores/my-data-store/sessions/5386462384953257772", "search_spec": { "search_params": { "filter": "doc_id: ANY(\"projects/7654321/locations/global/collections/default_collection/dataStores/my-data-store/branches/0/documents/3825eac51ef9e934bbc558faa42f4c71\")" } } }'
{ "answer": { "name": "projects/7654321/locations/global/collections/default_collection/dataStores/my-data-store/sessions/5386462384953257772/answers/2365204301797191130", "state": "SUCCEEDED", "answerText": "There is no comparison between BigQuery and Spanner database because they are not the same thing. BigQuery is a cloud-based data warehouse that is used to store and analyze large amounts of data. Spanner is a distributed database system that is designed to handle large amounts of data and to be highly available.", "steps": [ { "state": "SUCCEEDED", "description": "Rephrase the query and search.", "actions": [ { "searchAction": { "query": " What is Spanner?" }, "observation": { "searchResults": [ { "document": "projects/7654321/locations/global/collections/default_collection/dataStores/my-data-store/branches/0/documents/9deefb55a85b2e8052772a7e2039c3d8", "uri": "gs://my-bucket-123/documents/4116ef89f18439458b317da3fdaed746.txt", "chunkInfo": [ { "content": "Tools that are used in particular fields or activities may have different classification such as \"instrument\", \"utensil\", \"implement\", \"machine\", \"device,\" or \"apparatus\".The set of tools needed to achieve a goal is \"apparatus \".The knowledge of constructing, obtaining and using tools is technology.Hand tools have been used by humans since the early age when stones were used for hammering and cutting.During the early age tools were made by casting the copper and tin alloys.Bronze tools were sharper and harder than those made of stone.Later the bronze was replaced by iron, and tools became even stronger and more durable.A wrench or spanner is a tool used to provide grip and mechanical advantage in twisting objects, , like nuts and bolts or keep them from turning.The most common architecture are called open ended spanner and ring spanner.The term wrench is commonly used for tools that turn non fastening equipment's.A Spanner refers to a particular wrench with a series of pins or tabs around the circumference.These pins fit into the holes cut into the object to be turned.The most common shapes are called open end wrench and box end wrench.This Web site will let you go through the products manufactured by us." } ] }, { "document": "projects/7654321/locations/global/collections/default_collection/dataStores/my-data-store/branches/0/documents/945d6b30f28873f7f7df1ae04db5639f", "uri": "gs://my-bucket-123/documents/c46991fd4ed7f71998b92cd8fb823435.txt", "chunkInfo": [ { "content": "Recently, Google Fellow Jeff Dean gave a revealing talk on large-scale computing systems in which he discussed technical details of a new storage and computation system called Spanner, which is designed for up to 10 million servers.Skynet, anyone?All those hardware must use a lot of electricity (indeed, Googleville data center is calculated to require about 103 megawatts of electricity - enough to power 82,000 homes or a city the size of Tacoma, Washington), but just how much energy do you use when you perform a Google search?Google calculated that it uses about 1 kJ (0.0003 kWh) of energy to answer the average search query.It's so efficient that your PC will likely use more energy in the time it takes to do a Google search.To offset its electricity consumption, Google even installed 1.6MW solar panels on the rooftops of the Googleplex.A total of 9,212 solar panels generate 4,475 kWh daily, the equivalent of about the amount of electricity used by 1,000 California homes.\"I began thinking about building a bicycle-based Street View system after realizing how many interesting places around the world - ranging from historic landmarks to beautiful trails to shopping districts - aren't accessible by car,\" says Dan." } ] }, { "document": "projects/7654321/locations/global/collections/default_collection/dataStores/my-data-store/branches/0/documents/39e8b006e2f01ddd5557fe274220cf08", "uri": "gs://my-bucket-123/documents/1c7825927b7e0d7df27bdd5dd731871b.txt", "chunkInfo": [ { "content": "Most people refer to slotted screw heads as flathead screws, in part because there is one narrow opening made for a flat screwdriver.It is a very inexpensive and common type of screw head, but unfortunately, the screw is also very prone to stripping.However, to prevent you from over-tightening them, slotted screws are actually supposed to strip.The screws are best for projects that require only a few screws that you can screw in manually instead of with a powered drill.This, in turn, often causes the screw to cam out or strip.Usually shaped like a hexagon or round-shaped, these screws look like they have a little cap or hat on top of the head, and pictures of them can be found at this link.Cap screws come in various lengths and materials, but each of them has heads that look like they're wearing a cap.The spanner screw has two rectangular pins, often called its "snake eyes."Used very infrequently in industry and trade, the spanner screw does have one advantage, and that is the screw head's surface is only broken by the two small pins, but otherwise, it is very solid.Furthermore, the spanner screw prevents over-tightening of the screw itself." } ] }, { "document": "projects/7654321/locations/global/collections/default_collection/dataStores/my-data-store/branches/0/documents/39e8b006e2f01ddd5557fe274220cf08", "uri": "gs://my-bucket-123/documents/1c7825927b7e0d7df27bdd5dd731871b.txt", "chunkInfo": [ { "content": "High-stress concentrations and force transmission go together with this type of screw, but because of the two pins, there is less stress on the individual faces.If you wish to transmit high torque, the spanner screw is not one you should use.Also called Robertson's screws, square recess screw heads have a square center point that helps you avoid cam-outs.The bit you use to drive square recess screw heads also juts out on a square taper, creating a self-holding design so that you won't have to hold the bit in place, which is very convenient, regardless of the project at hand.Double-square drives, which have two of Robertson's squares forming an 8-point star in the very middle.For higher torque application, you can use a Robertson's bit to drive it, or even a special one for the double-square.The triple-square drive, which consists of three Robertson's squares and creates a 12-point star.These are usually used when you need a lot of force but do not want to strip the screw.You can find triple-square drives on drivetrain components and internal car parts, among other items found on a vehicle." } ] }, { "document": "projects/7654321/locations/global/collections/default_collection/dataStores/my-data-store/branches/0/documents/39e8b006e2f01ddd5557fe274220cf08", "uri": "gs://my-bucket-123/documents/1c7825927b7e0d7df27bdd5dd731871b.txt", "chunkInfo": [ { "content": "A few types have built-in flanges, while others have just a full head that is in the hexagon shape.To remove or install these types of screws, you need a socket or a wrench.Since the full head is turned during the action, instead of just the internal portion of the head, you can get great leverage on this type of screw head.A hex internal screw head requires an Allen wrench to remove or install it.They are very common for furniture that requires some assembly, as they are less likely to be damaged by an Allen wrench when you're installing the screws.If you were to use a slotted or Phillips screw, there may be some damage to the furniture, which no one wants.Hex internal screw heads have an indented shape on the top, which is why they are called internal screw heads.With 12 small teeth that are much finer than those in the TORX, this type of screw has great force distribution.It is similar to the TORX screw in that there is a large area for the driving force, allowing for transferring of high torques.The multi-tooth screw is used frequently in the automotive industry, such as when high torques are needed in order to transfer high-tensile force.It is also a good screw to prevent unauthorized tampering with the screws themselves." } ] } ] } }, { "searchAction": { "query": " What is BigQuery?" }, "observation": { "searchResults": [ { "document": "projects/7654321/locations/global/collections/default_collection/dataStores/my-data-store/branches/0/documents/d993d922043374f5ef7ba297c158b106", "uri": "gs://my-bucket-123/documents/058dee0ec23a3e92f9bfd7cd29840e8f.txt", "chunkInfo": [ { "content": "There are a lot of Databases available in the market such as MS Access, Oracle and many others.For you to write programs that interact with these databases easily, there has to be a way where you could get information from all these databases using the same method.For this purpose SQL was developed.It is a kind of language (simple when compared to the likes of C or C++) which enables you to ask all your queries to a database without bothering about the exact type of database.When you use this Query the database engine would first find the table called people.Then it would find a column called firstname.Next it would compare all the values in that column with 'Reena'.Finally it would return all the details wherever it finds a match for the firstname.When you write a database program in VC++ or Java or any other language for that matter, you would make a database connection to your database and then you would query the database using SQL queries.When you query the database with any SQL query the database returns a recordset.A recordset is basically a set of records (all the entries that your query returns).This recordset is received in your program and all languages have a data structure which represents a recordset." } ] }, { "document": "projects/7654321/locations/global/collections/default_collection/dataStores/my-data-store/branches/0/documents/3825eac51ef9e934bbc558faa42f4c71", "uri": "gs://my-bucket-123/documents/26f5872b0719790cb966a697bfa1ea27.txt", "chunkInfo": [ { "content": "Inner join creates a new result table by combining column values of two tables (A and B)based upon the join-predicate.The query compares each row of A with each row of B to find all pairs of rows which satisfy the join-predicate.When the join-predicate is satisfied, column values for each matched pair of rows of A and B are combined into a result row.The result of the join can be defined as the outcome of first taking the Cartesian product (or cross-join) of all records in the tables (combining every record in table A with every record in table B)—then return all records which satisfy the join predicate.Current SQL implementations normally use other approaches like a hash join or a sort-merge join where possible, since computing the Cartesian product is very inefficient.The \"implicit join notation\" simply lists the tables for joining (in the FROM clause of the SELECT statement), using commas to separate them.Thus, it specifies a cross-join, and the WHERE clause may apply additional filter-predicates (which function comparably to the join-predicates in the explicit notation).INSERT allows one to insert new rows into a table.One can insert a single row at a time or several rows as a result of a query." } ] }, { "document": "projects/7654321/locations/global/collections/default_collection/dataStores/my-data-store/branches/0/documents/3825eac51ef9e934bbc558faa42f4c71", "uri": "gs://my-bucket-123/documents/26f5872b0719790cb966a697bfa1ea27.txt", "chunkInfo": [ { "content": "The Structured Query Language (SQL) is a database management programming language.SQL is a tool for accessing databases, and more specifically, relational databases, and can be used with different database products.This chapter will prepare you to learn basic database management using this language.SQLite – To implement SQL as a library, you need SQLite.SQLite is intended to provide users and programs a way to store data using a SQL interface within the program.SQLite3 can be used to manipulate SQLite databases for major Linux distros.SQL is used to access relational databases.Each database contains more or less tables which in turn contain more or less rows and columns.Hereby a single row is seen as a separate object with features represented by the tables' columns.To access a table's data you first have to connect to its database.With the same table, the query SELECT * FROM T WHERE C1 = 1 will result in all the elements of all the rows where the value of column C1 is '1' being shown.A WHERE clause specifies that a SQL statement should only affect rows that meet specified criteria.The criteria are expressed in the form of predicates.WHERE clauses are not mandatory clauses of SQL statements, but should be used to limit the number of rows affected by a SQL DML statement or returned by a query." } ] }, { "document": "projects/7654321/locations/global/collections/default_collection/dataStores/my-data-store/branches/0/documents/d993d922043374f5ef7ba297c158b106", "uri": "gs://my-bucket-123/documents/058dee0ec23a3e92f9bfd7cd29840e8f.txt", "chunkInfo": [ { "content": "Once this data structure (in your program) gets populated with the results from the database query, your could use a for loop to loop through all the entries.When you connect to a database and execute SQL Queries,the results of the Query are returned back to your program.This returned data has to be stored in some Object or Data Structure within your program to be used by your program.Once you store the results in this Object you no longer have to be connected to the Database.For a more detailed explanation please refer to a book on Database programming.The SELECT statement lets you select a set of values from a table in a database.The values selected from the database table would depend on the various conditions that are specified in the SQL query.Here are some ways to use the SELECT statement.I have listed the SQL statements and the respective results that you would obtain if you would execute those queries.Note : That whenever you are comparing a varchar the value should be enclosed in single inverted commas ( ' ).Whereas when you compare an integer the value need not be enclosed in single inverted commas.Good news!!Yes you can..Actually as a matter of fact, once you start developing professional database applications you would almost always use such Boolean operators to make effective queries." } ] }, { "document": "projects/7654321/locations/global/collections/default_collection/dataStores/my-data-store/branches/0/documents/d993d922043374f5ef7ba297c158b106", "uri": "gs://my-bucket-123/documents/058dee0ec23a3e92f9bfd7cd29840e8f.txt", "chunkInfo": [ { "content": "Remember that when you use the COUNT keyword you do not get the actual lastname of the persons but you only get the total number of records that exist in the database that match your requirements.And in this case since DISTINCT was also used it would find the total number of records where there are distinct firstname only.Ok here are 2 new things that I have used only a few times in my programs.But they maybe useful to you,so I shall talk about them.There are 2 keywords called GROUP BY and HAVING.The GROUP BY keywords have been added to SQL because aggregate functions (like SUM) return the aggregate of all column values every time they are called.Without the GROUP BY functionality, finding the sum for each individual group of column values was not possible.Returns a recordset with 3 records.Each record has 2 values.The first record would have the value 'Sega' and '85000'.The second record would have the values ' Microsoft' and '85000'.And the third record would have the values 'Sega' and '85000'.Thus it is clear that this is not what was required.There is no sense in getting the sum of all the profits of all the companies along with each company name." } ] } ] } }, { "searchAction": { "query": " What are the advantages and disadvantages of BigQuery?" }, "observation": { "searchResults": [ { "document": "projects/7654321/locations/global/collections/default_collection/dataStores/my-data-store/branches/0/documents/031b1c5b3104a11802bf127ede7443f0", "uri": "gs://my-bucket-123/documents/976bd7e6aceddcc0bd4f5eac55896a7a.txt", "chunkInfo": [ { "content": "It was reviewed that the extended version of Mapreduce for more data intensive applications such as HaLoop and Spark Map Reduce work well for Iterative computation.DT Editorial Services, Black Book 6 states that Hadoop is an open-source platform that provides analytical technologies and computational power required to work with such large volumes of data.A Hadoop cluster consists of single MasterNode and multiple SlaveNodes.The master node consists of a NameNode and a JobTracker.The replica node or worker node acts as both a DataNode and a TaskTracker.Hadoop MapReduce is a computational framework used in hadoop to perform all the mathematical computations.It is based on a parallel and distributed implementation of MapReduce algorithm that provides high performance.P. Sudha et al. 63 proposed that Big data has to deal with large and complex dataset that can be structured, semi-structured or unstructured and will typically not fit into memory to be processed.MapReduce is a programming model for processing large datasets distributed on large clusters.MapReduce framework is basically designed to compute data demanding applications to support effective decision making.Since its introduction, remarkable research efforts have been put to make it more familiar to the users subsequently utilized to support the execution of enormous data intensive applications.This survey paper highlights and investigates various applications using recent MapReduce models." } ] }, { "document": "projects/7654321/locations/global/collections/default_collection/dataStores/my-data-store/branches/0/documents/031b1c5b3104a11802bf127ede7443f0", "uri": "gs://my-bucket-123/documents/976bd7e6aceddcc0bd4f5eac55896a7a.txt", "chunkInfo": [ { "content": "Praveen T et al. 41 proposed that huge amount of data is being generated with the arrival of new updates in various fields like social networks, e-commerce, finance and education, etc.The retrieval of data becomes complex requiring efficient algorithms to manage the tasks.Map reduce framework involves task-level computation which is used in the analysis of big data.This involves re-computation which results in long time processing and less performance.Hence, this framework seems to be less efficient.Therefore, Bipartite graph is used to perform Map reduce for efficient and fast processing.Also, the main focus is on reducing time complexity to make the system competent by processing large data sets using Bipartite graph.M. Senthilkumar et al. 67 proposed that Big Data Applications with Scheduling becomes an active research area in past years.The Hadoop framework becomes very popular and most used frameworks in a distributed data processing.Various scheduling algorithms of the MapReduce model using Hadoop vary with design and behavior and are used for handling many issues like data locality, awareness with resource, energy and time.This paper gives the outline of job scheduling, classification of the scheduler.To overcome the scheduling issues many job scheduling algorithms are presented: FIFO scheduler, Fair scheduler, Capacity scheduler and Deadline – Constraint scheduler.The advantages and disadvantages of respective algorithms are discussed." } ] }, { "document": "projects/7654321/locations/global/collections/default_collection/dataStores/my-data-store/branches/0/documents/031b1c5b3104a11802bf127ede7443f0", "uri": "gs://my-bucket-123/documents/976bd7e6aceddcc0bd4f5eac55896a7a.txt", "chunkInfo": [ { "content": "The algorithm maximizes the number of jobs that can be run on a cluster satisfying time requirements of all running jobs.The future work may include Deadline – Constraint scheduling algorithm validation using tool and experiments may be constructed for the same for a cluster of nodes processing real-time data and reducing computational time spent while waiting for a map or reduce slot.1Dr.M. Moorthy, R. Baby and S. Senthamaraiselvi, "An Analysis for Big Data and its Technologies", International Journal of Computer Science Engineering and Technology (IJCSET), Vol. 04, Issue 12, pp. 412-418, 2014.2A. A.Pandagale and A. R.Surve, "Big Data Analysis Using Hadoop Framework", International Journal of Research and Analytical Reviews (IJRAR), Vol. 03, Issue 01, pp. 87-91, 2016.5H.S. Bhosale and Prof. D. P." } ] }, { "document": "projects/7654321/locations/global/collections/default_collection/dataStores/my-data-store/branches/0/documents/031b1c5b3104a11802bf127ede7443f0", "uri": "gs://my-bucket-123/documents/976bd7e6aceddcc0bd4f5eac55896a7a.txt", "chunkInfo": [ { "content": "In dynamic scheduling, allocation of jobs to the processors is done during execution time.A little basic knowledge is known about the resource needs of a job.It is also unknown in what type of environment the job will execute during its lifetime.Decision is made when a job begins its execution in the dynamic environment of the system.Deadline – Constraint scheduler schedules jobs based on the deadline constraints mentioned by users.This type of algorithms ensures that the jobs whose deadlines can be met are scheduled for execution.It mainly focuses on increasing system utilization.In this chapter, scheduling algorithms and its various categories were discussed.There are mainly two categories: Static schedulers and Dynamic schedulers which are further categorized into FIFO, FAIR, Capacity and Deadline – Constraint schedulers respectively.In this chapter, Static Schedulers are described extensively.The different types of static scheduling algorithms: FIFO, FAIR and Capacity are deeply discussed.Also, the software description is given and the background of the algorithms along with their scope, advantages and disadvantages are described in this chapter.Apache Hadoop is a collection of open-source software utilities that facilitate using a network of many computers to solve problems involving massive amounts of data and computation 21.It also provides a software framework for distributed storage and processing of big data using the MapReduce programming model." } ] }, { "document": "projects/7654321/locations/global/collections/default_collection/dataStores/my-data-store/branches/0/documents/031b1c5b3104a11802bf127ede7443f0", "uri": "gs://my-bucket-123/documents/976bd7e6aceddcc0bd4f5eac55896a7a.txt", "chunkInfo": [ { "content": "The backgrounds, examples, algorithms, advantages and disadvantages of different schedulers are also explained.Deadline – Constraint scheduler schedules jobs based on the deadline constraints mentioned by users 31.This type of algorithm ensures that the jobs whose deadlines can be met are scheduled for execution.It mainly focuses on increasing system utilization.In this scheduling strategy 32, 33, the user specified deadline constraints at the time of scheduling the jobs to ensure that the jobs scheduled for execution meets the deadline 34,35. It deals with the deadline requirement by the cost model of job execution, which considers parameters such as input size of data, data distribution, map and reduce runtime etc.Whenever any job is scheduled, it is checked by the scheduler whether it will be completed within the time specified by the deadline or not.When a job is submitted 36, schedulability test is performed to determine whether the job can be finished within the specified deadline or not.A job is schedulable, if the minimum number of tasks for both map and reduce is less than or equal to the number of available slots.This scheduler shows that when a deadline for job is different, then the scheduler assigns different number of tasks to TaskTracker and makes sure that the specified deadline is met.Deadline – Constraint scheduler has high resource utilization." } ] } ] } } ] } ] }, "session": { "name": "projects/7654321/locations/global/collections/default_collection/dataStores/my-data-store/sessions/5386462384953257772", "state": "IN_PROGRESS", "userPseudoId": "test_user", "turns": [ { "query": { "queryId": "projects/7654321/locations/global/questions/630", "text": "Compare bigquery with spanner database?" }, "answer": "projects/7654321/locations/global/collections/default_collection/dataStores/my-data-store/sessions/5386462384953257772/answers/2365204301797191130" } ], "startTime": "2024-04-03T05:31:30.217535Z", "endTime": "1970-01-01T00:00:00Z" } }Repeat step 2 for each new query in the session.
Delete a session from the data store
The following command shows how to call the answer method and
delete a session from the data store.
By default, sessions older than 60 days are automatically deleted. However, if you want to delete a particular session—for example, if it contains sensitive content, then use this API call to delete it.
REST
To delete a session from a data store, do the following:
Run the following curl command:
curl -X POST -H "Authorization: Bearer $(gcloud auth print-access-token)" \ -H "Content-Type: application/json" \ "https://discoveryengine.googleapis.com/v1beta/projects/PROJECT_ID/locations/global/collections/default_collection/dataStores/DATA_STORE_ID/sessions/SESSION_ID"
Replace the following:
PROJECT_ID: the ID of your Google Cloud project.DATA_STORE_ID: the ID of the Vertex AI Search data store.SESSION_ID: the ID of the session that you want to delete.
Update a session
There are various reasons that you might want to update a session. For example, to do one of the following:
- Mark a session as completed
- Merge the messages from one session into another
- Change a user's pseudo ID
The following command shows how to call the answer method and
update a session in the data store.
REST
To update a session from a data store, do the following:
Run the following curl command:
curl -X PATCH \ -H "Authorization: Bearer $(gcloud auth print-access-token)" \ -H "Content-Type: application/json" \ "https://discoveryengine.googleapis.com/v1beta/projects/PROJECT_ID/locations/global/collections/default_collection/dataStores/DATA_STORE_ID/sessions/SESSION_ID?updateMask=state" \ -d '{ "state": "NEW_STATE" }'Replace the following:
PROJECT_ID: the ID of your Google Cloud project.DATA_STORE_ID: the ID of the Vertex AI Search data store.SESSION_ID: the ID of the session that you want to update.NEW_STATE: The new value for the state—for example,COMPLETED.
List all sessions
The following command shows how to call the answer method and
list the sessions in the data store.
REST
To list the sessions in a data store, do the following:
Run the following curl command:
curl -X GET \ -H "Authorization: Bearer $(gcloud auth print-access-token)" \ -H "Content-Type: application/json" \ "https://discoveryengine.googleapis.com/v1beta/projects/PROJECT_ID/locations/global/collections/default_collection/dataStores/DATA_STORE_ID/sessions"
Replace the following:
PROJECT_ID: the ID of your Google Cloud project.DATA_STORE_ID: the ID of the Vertex AI Search data store.
List sessions for a user
The following command shows how to call the answer method to
list sessions associated with a user or visitor.
REST
To list sessions associated with a user or visitor, do the following:
Run the following curl command:
curl -X GET \ -H "Authorization: Bearer $(gcloud auth print-access-token)" \ -H "Content-Type: application/json" \ "https://discoveryengine.googleapis.com/v1beta/projects/PROJECT_ID/locations/global/collections/default_collection/dataStores/DATA_STORE_ID/sessions?filter=user_pseudo_id=USER_PSEUDO_ID"
Replace the following:
PROJECT_ID: the ID of your Google Cloud project.DATA_STORE_ID: the ID of the Vertex AI Search data store.USER_PSEUDO_ID: the pseudo ID of the user whose sessions you want to list.
List sessions for a user and state
The following command shows how to call the answer method to
list sessions in a given state for a particular user.
REST
To list sessions for a user that are open or closed and associated with a given user or visitor, do the following:
Run the following curl command:
curl -X GET -H "Authorization: Bearer $(gcloud auth print-access-token)" \ -H "Content-Type: application/json" \ "https://discoveryengine.googleapis.com/v1beta/projects/PROJECT_ID/locations/global/collections/default_collection/dataStores/DATA_STORE_ID/sessions?filter=user_pseudo_id=USER_PSEUDO_ID%20AND%20state=STATE"
Replace the following:
PROJECT_ID: the ID of your Google Cloud project.DATA_STORE_ID: the ID of the Vertex AI Search data store.USER_PSEUDO_ID: the pseudo ID of the user whose sessions you want to list.STATE: the state of the session:STATE_UNSPECIFIED(closed or unknown) orIN_PROGRESS(open).