Mistral AI models

Mistral AI models on Vertex AI offer fully managed and serverless models as APIs. To use a Mistral AI model on Vertex AI, send a request directly to the Vertex AI API endpoint. Because Mistral AI models use a managed API, there's no need to provision or manage infrastructure.

You can stream your responses to reduce the end-user latency perception. A streamed response uses server-sent events (SSE) to incrementally stream the response.

You pay for Mistral AI models as you use them (pay as you go). For pay-as-you-go pricing, see Mistral AI model pricing on the Vertex AI pricing page.

To see an example of getting started with Mistral AI models on Vertex AI, run the "Getting Started with Mistral AI Models" Jupyter notebook in one of the following environments:

Open in Colab | Open in Colab Enterprise | Open in Vertex AI Workbench user-managed notebooks | View on GitHub

Available Mistral AI models

The following models are available from Mistral AI to use in Vertex AI. To access a Mistral AI model, go to its Model Garden model card.

Mistral Large (2407)

Mistral Large (2407) is Mistral AI's flagship model for text generation. It reaches top-tier reasoning capabilities and can be used for complex multilingual tasks, including text understanding, transformation, and code generation. For more information, see Mistral AI's post about Mistral Large (2407).

Mistral Large (2407) stands out on the following dimensions:

• Multi-lingual by design. Dozens of languages supported, including English, French, German, Spanish, Italian, Chinese, Japanese, Korean, Portuguese, Dutch and Polish.
• Proficient in coding. Trained on 80+ coding languages such as Python, Java, C, C++, JavaScript, and Bash. Also trained on more specific languages such as Swift and Fortran.
• Agent-centric. Best-in-class agentic capabilities with native function calling and JSON outputting.
• Advanced Reasoning. State-of-the-art mathematical and reasoning capabilities.

Go to the Mistral Large (2407) model card

Mistral Nemo

Mistral Nemo is Mistral AI's most cost efficient proprietary model. It is the ideal choice for low-latency workloads and simple tasks that can be done in bulk, such as classification, customer support, and text generation. For more information, see Mistral AI's documentation.

Mistral Nemo is optimized for the following use cases:

• Generating and classifying text.
• Building agents for customer support scenarios.
• Generating code, completion, review, and comments. Supports all mainstream coding languages.

Go to the Mistral Nemo model card

Codestral

Codestral is a generative model that is specifically designed and optimized for code generation. The model can help developers write and interact with code through a shared instruction and completion API endpoint. As it masters code and English, you can use Codestral to design advanced AI applications for software developers.

Codestral is optimized for the following use cases:

• Generating code, providing code completion, suggestions, and translation.
• Understanding your code to provide a summary and explanation of your code.
• Reviewing the quality of your code by helping refactor your code, fixing bugs, and generating test cases.

Go to the Codestral model card

Use Mistral AI models

When you send requests to use Mistral AI's models, use the following model names:

• For Mistral Large (2407), use mistral-large@2407.
• For Mistral Nemo, use mistral-nemo@2407.
• For Codestral, use codestral@2405.

We recommend that you use the model versions that include a suffix that starts with an @ symbol because of the possible differences between model versions. If you don't specify a model version, the latest version is always used, which can inadvertently affect your workflows when a model version changes.

Note: The maximum allowed image file size is 5 MB and you can include up to 20 images in one request.

Before you begin

To use Mistral AI models with Vertex AI, you must perform the following steps. The Vertex AI API (aiplatform.googleapis.com) must be enabled to use Vertex AI. If you already have an existing project with the Vertex AI API enabled, you can use that project instead of creating a new project.

Make sure you have the required permissions to enable and use partner models. For more information, see Grant the required permissions.

1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

2. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

   Go to project selector

3. Make sure that billing is enabled for your Google Cloud project.

4. Enable the Vertex AI API.

   Enable the API

5. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

   Go to project selector

6. Make sure that billing is enabled for your Google Cloud project.

7. Enable the Vertex AI API.

   Enable the API

8. Go to one of the following Model Garden model cards, then click enable:
   • Go to the Mistral Large (2407) model card
   • Go to the Mistral Nemo model card
   • Go to the Codestral model card

Make a streaming call to a Mistral AI model

The following sample makes a streaming call to a Mistral AI model.

REST

After you set up your environment, you can use REST to test a text prompt. The following sample sends a request to the publisher model endpoint.

Before using any of the request data, make the following replacements:

• LOCATION: A region that supports Mistral AI models.
• MODEL: The model name you want to use. In the request body, exclude the @ model version number.
• ROLE: The role associated with a message. You can specify a user or an assistant. The first message must use the user role. The models operate with alternating user and assistant turns. If the final message uses the assistant role, then the response content continues immediately from the content in that message. You can use this to constrain part of the model's response.
• STREAM: A boolean that specifies whether the response is streamed or not. Stream your response to reduce the end-use latency perception. Set to true to stream the response and false to return the response all at once.
• CONTENT: The content, such as text, of the user or assistant message.
• MAX_OUTPUT_TOKENS: Maximum number of tokens that can be generated in the response. A token is approximately 3.5 characters. 100 tokens correspond to roughly 60-80 words.

  Specify a lower value for shorter responses and a higher value for potentially longer responses.

HTTP method and URL:

POST https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/publishers/mistral/models/MODEL:streamRawPredict

Request JSON body:

{
"model": MODEL,
  "messages": [
   {
    "role": "ROLE",
    "content": "CONTENT"
   }],
  "max_tokens": MAX_TOKENS,
  "stream": true
}

To send your request, choose one of these options:

curl

Note: The following command assumes that you have logged in to the gcloud CLI with your user account by running gcloud init or gcloud auth login , or by using Cloud Shell, which automatically logs you into the gcloud CLI . You can check the currently active account by running gcloud auth list.

Save the request body in a file named request.json, and execute the following command:

curl -X POST \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     -H "Content-Type: application/json; charset=utf-8" \
     -d @request.json \
     "https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/publishers/mistral/models/MODEL:streamRawPredict"

PowerShell

Note: The following command assumes that you have logged in to the gcloud CLI with your user account by running gcloud init or gcloud auth login . You can check the currently active account by running gcloud auth list.

Save the request body in a file named request.json, and execute the following command:

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method POST `
    -Headers $headers `
    -ContentType: "application/json; charset=utf-8" `
    -InFile request.json `
    -Uri "https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/publishers/mistral/models/MODEL:streamRawPredict" | Select-Object -Expand Content

You should receive a JSON response similar to the following.

Response

data: {
    "id": "0e9c8e69e5924f729b39bc60bac9e0be",
    "object": "chat.completion.chunk",
    "created": 1720807292,
    "model": "MODEL",
    "choices": [
        {
            "index": 0,
            "delta": {
              "content": "OUTPUT"
            },
            "finish_reason": null,
            "logprobs": null
        }
    ]
}

data: {
    "id": "0e9c8e69e5924f729b39bc60bac9e0be",
    "object": "chat.completion.chunk",
    "created": 1720807292,
    "model": "MODEL",
    "choices": [
        {
            "index": 0,
            "delta": {
              "content": "OUTPUT"
            },
            "finish_reason": null,
            "logprobs": null
        }
    ]
}
...

Make a unary call to a Mistral AI model

The following sample makes a unary call to a Mistral AI model.

REST

After you set up your environment, you can use REST to test a text prompt. The following sample sends a request to the publisher model endpoint.

Before using any of the request data, make the following replacements:

• LOCATION: A region that supports Mistral AI models.
• MODEL: The model name you want to use. In the request body, exclude the @ model version number.
• ROLE: The role associated with a message. You can specify a user or an assistant. The first message must use the user role. The models operate with alternating user and assistant turns. If the final message uses the assistant role, then the response content continues immediately from the content in that message. You can use this to constrain part of the model's response.
• STREAM: A boolean that specifies whether the response is streamed or not. Stream your response to reduce the end-use latency perception. Set to true to stream the response and false to return the response all at once.
• CONTENT: The content, such as text, of the user or assistant message.
• MAX_OUTPUT_TOKENS: Maximum number of tokens that can be generated in the response. A token is approximately 3.5 characters. 100 tokens correspond to roughly 60-80 words.

  Specify a lower value for shorter responses and a higher value for potentially longer responses.

HTTP method and URL:

POST https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/publishers/mistral/models/MODEL:rawPredict

Request JSON body:

{
"model": MODEL,
  "messages": [
   {
    "role": "ROLE",
    "content": "CONTENT"
   }],
  "max_tokens": MAX_TOKENS,
  "stream": false
}

To send your request, choose one of these options:

curl

Note: The following command assumes that you have logged in to the gcloud CLI with your user account by running gcloud init or gcloud auth login , or by using Cloud Shell, which automatically logs you into the gcloud CLI . You can check the currently active account by running gcloud auth list.

Save the request body in a file named request.json, and execute the following command:

curl -X POST \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     -H "Content-Type: application/json; charset=utf-8" \
     -d @request.json \
     "https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/publishers/mistral/models/MODEL:rawPredict"

PowerShell

Note: The following command assumes that you have logged in to the gcloud CLI with your user account by running gcloud init or gcloud auth login . You can check the currently active account by running gcloud auth list.

Save the request body in a file named request.json, and execute the following command:

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method POST `
    -Headers $headers `
    -ContentType: "application/json; charset=utf-8" `
    -InFile request.json `
    -Uri "https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/publishers/mistral/models/MODEL:rawPredict" | Select-Object -Expand Content

You should receive a JSON response similar to the following.

Response

{
    "id": "e71d13ffb77344a08e34e0a22ea84458",
    "object": "chat.completion",
    "created": 1720806624,
    "model": "MODEL",
    "choices": [
        {
            "index": 0,
            "message": {
                "role": "assistant",
                "content": "OUTPUT",
                "tool_calls": null
            },
            "finish_reason": "stop",
            "logprobs": null
        }
    ],
    "usage": {
        "prompt_tokens": 17,
        "total_tokens": 295,
        "completion_tokens": 278
    }
}

Mistral AI model region availability and quotas

Important: Machine learning (ML) processing for all available Mistral AI models occurs within the US when requests are made to regionally-available APIs in the US, or within the EU when requests are made to regionally-available APIs in Europe.

For Mistral AI models, a quota applies for each region where the model is available. The quota is specified in queries per minute (QPM) and tokens per minute (TPM). TPM includes both input and output tokens.

The supported regions, default quotas, and maximum context length for each Mistral AI model is listed in the following tables:

Mistral Large (2407)

Region	Quota system	Supported context length
us-central1	60 QPM, 200,000 TPM	128,000 tokens
europe-west4	60 QPM, 200,000 TPM	128,000 tokens

Mistral Nemo

Region	Quota system	Supported context length
us-central1	60 QPM, 200,000 TPM	128,000 tokens
europe-west4	60 QPM, 200,000 TPM	128,000 tokens

Codestral

Region	Quota system	Supported context length
us-central1	60 QPM, 200,000 TPM	32,000 tokens
europe-west4	60 QPM, 200,000 TPM	32,000 tokens

If you want to increase any of your quotas for Generative AI on Vertex AI, you can use the Google Cloud console to request a quota increase. To learn more about quotas, see Work with quotas.