To design a prompt that works well, test different versions of the prompt and experiment with prompt parameters to determine what results in the optimal response. You can test prompts programmatically with the Codey APIs and in the Google Cloud console with Vertex AI Studio.
Test code completion prompts
To test code completion prompts, choose one of the following methods.
REST
To test a code completion prompt with the Vertex AI API, send a POST request to the publisher model endpoint.
Before using any of the request data, make the following replacements:
- PROJECT_ID: Your project ID.
- PREFIX:
For code models,
prefix
represents the beginning of a piece of meaningful programming code or a natural language prompt that describes code to be generated. The model attempts to fill in the code in between theprefix
andsuffix
. - SUFFIX:
For code completion,
suffix
represents the end of a piece of meaningful programming code. The model attempts to fill in the code in between theprefix
andsuffix
. - TEMPERATURE:
The temperature is used for sampling during response generation. Temperature controls the degree of
randomness in token selection. Lower temperatures are good for prompts that require a less
open-ended or creative response, while higher temperatures can lead to more diverse or creative
results. A temperature of
0
means that the highest probability tokens are always selected. In this case, responses for a given prompt are mostly deterministic, but a small amount of variation is still possible. - MAX_OUTPUT_TOKENS:
Maximum number of tokens that can be generated in the response. A token is
approximately four characters. 100 tokens correspond to roughly 60-80 words.
Specify a lower value for shorter responses and a higher value for potentially longer responses.
- CANDIDATE_COUNT:
The number of response variations to return. For each request, you're charged for the
output tokens of all candidates, but are only charged once for the input tokens.
Specifying multiple candidates is a Preview feature that works with
generateContent
(streamGenerateContent
is not supported). The following models are supported:- Gemini 1.5 Flash:
1
-8
, default:1
- Gemini 1.5 Pro:
1
-8
, default:1
- Gemini 1.0 Pro:
1
-8
, default:1
int
between 1 and 4. - Gemini 1.5 Flash:
HTTP method and URL:
POST https://us-central1-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/us-central1/publishers/google/models/code-gecko:predict
Request JSON body:
{ "instances": [ { "prefix": "PREFIX", "suffix": "SUFFIX"} ], "parameters": { "temperature": TEMPERATURE, "maxOutputTokens": MAX_OUTPUT_TOKENS, "candidateCount": CANDIDATE_COUNT } }
To send your request, choose one of these options:
curl
Save the request body in a file named request.json
,
and execute the following command:
curl -X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
"https://us-central1-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/us-central1/publishers/google/models/code-gecko:predict"
PowerShell
Save the request body in a file named request.json
,
and execute the following command:
$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }
Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://us-central1-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/us-central1/publishers/google/models/code-gecko:predict" | Select-Object -Expand Content
You should receive a JSON response similar to the following.
Python
To learn how to install or update the Vertex AI SDK for Python, see Install the Vertex AI SDK for Python. For more information, see the Python API reference documentation.
Node.js
Before trying this sample, follow the Node.js setup instructions in the Vertex AI quickstart using client libraries. For more information, see the Vertex AI Node.js API reference documentation.
To authenticate to Vertex AI, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.
Java
Before trying this sample, follow the Java setup instructions in the Vertex AI quickstart using client libraries. For more information, see the Vertex AI Java API reference documentation.
To authenticate to Vertex AI, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.
Console
To test a code completion prompt using Vertex AI Studio in the Google Cloud console, do following :
- In the Vertex AI section of the Google Cloud console, go to Vertex AI Studio.
- Click Get started.
- Click Code prompt.
- In Model, select the model with the name that begins with
code-gecko
. A three digit number aftercode-gecko
indicates the version number of the model. For example,code-gecko@002
is the name of version two of the stable version of the code completion model. - In Prompt, enter a code completion prompt.
- Adjust Temperature and Token limit to experiment with how they affect the response. For more information, see Code completion model parameters.
- Click Submit to generate a response.
- Click Save if you want to save a prompt
- Click View code to see the Python code or a curl command for your prompt
Example curl command
MODEL_ID="code-gecko"
PROJECT_ID=PROJECT_ID
curl \
-X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json" \
https://us-central1-aiplatform.googleapis.com/v1/projects/${PROJECT_ID}/locations/us-central1/publishers/google/models/${MODEL_ID}:predict -d \
$"{
'instances': [
{ 'prefix': 'def reverse_string(s):',
'suffix': ''
}
],
'parameters': {
'temperature': 0.2,
'maxOutputTokens': 64,
'candidateCount': 1
}
}"
To learn more about prompt design for code completion, see Create prompts for code completion.
Stream response from code model
To view sample code requests and responses using the REST API, see Examples using the streaming REST API.
To view sample code requests and responses using the Vertex AI SDK for Python, see Examples using Vertex AI SDK for Python for streaming.
What's next
- Learn how to create code completion prompts.
- Learn how to create code generation prompts.
- Learn about responsible AI best practices and Vertex AI's safety filters.