Live API 支持通过会话实现低延迟的多模态互动。它使用会话记忆来保留和回忆会话中互动的信息。这样，模型就可以回忆起之前提供或讨论过的信息。预配吞吐量支持 Gemini 2.5 Flash with Live API 模型。如需详细了解 Live API（包括会话限制和功能），请参阅 Live API 参考文档。
计算 Live API 的吞吐量
使用 Live API 时，存储在会话内存中的令牌可用于后续向模型发出的请求。因此，预配吞吐量会同时考虑传入令牌和同一请求中的会话内存令牌。这可能会导致每个请求处理的令牌数量大于用户在当前请求中发送的令牌数量。
Live API 对可存储在会话内存中的令牌总数有限制，并且还包含一个元数据字段，其中包含令牌总数。在计算处理请求所需的吞吐量时，您必须考虑会话内存中的令牌。如果您曾使用过随用随付 (PayGo) 的 Live API，则可以使用这些流量模式和会话令牌来帮助估算您的预配吞吐量需求。
[[["易于理解","easyToUnderstand","thumb-up"],["解决了我的问题","solvedMyProblem","thumb-up"],["其他","otherUp","thumb-up"]],[["很难理解","hardToUnderstand","thumb-down"],["信息或示例代码不正确","incorrectInformationOrSampleCode","thumb-down"],["没有我需要的信息/示例","missingTheInformationSamplesINeed","thumb-down"],["翻译问题","translationIssue","thumb-down"],["其他","otherDown","thumb-down"]],["最后更新时间 (UTC)：2025-09-04。"],[],[],null,["# Provisioned Throughput for Live API\n\n| **Request access:** For information about access to this release, see the [access request page](https://docs.google.com/forms/d/e/1FAIpQLScxBeD4UJ8GbUfX4SXjj5a1XJ1K7Urwvb0iSGdGccNcFRBrpQ/viewform).\n\nThis section explains how Provisioned Throughput works with the\nLive API for token counting and quota enforcement.\n\nThe Live API supports low-latency multimodal interactions through\nsessions. It uses a session memory to retain and recall information from\ninteractions within a session. This lets the model recall previously provided or discussed information. Provisioned Throughput supports\nthe Gemini 2.5 Flash with Live API model. For more\ninformation about the Live API, including session limits and\ncapabilities, see the\n[Live API reference](/vertex-ai/generative-ai/docs/model-reference/multimodal-live).\n\nCalculate throughput for Live API\n---------------------------------\n\nWhile using the Live API, the tokens stored in the session memory\ncan be used in subsequent requests to the model. As a result, Provisioned Throughput\ntakes into account the incoming tokens as well as session memory tokens in the\nsame request. This might lead to the number of tokens being processed per request\nbeing greater than the tokens sent by the user in the ongoing request.\n\nThe Live API has a limit on the total tokens that can be stored in\nthe session memory and also has a metadata field containing the total number\nof tokens. While calculating how much throughput is needed to serve your requests,\nyou must account for tokens in the session memory.\nIf you've used the Live API with pay-as-you-go (PayGo), you can\nuse these traffic patterns and session tokens to help estimate your\nProvisioned Throughput needs.\n\n### Example of how to estimate your Provisioned Throughput requirements for Live API\n\nDuring a session, all traffic is processed either as\nProvisioned Throughput or pay-as-you-go. If you reach your\nProvisioned Throughput quota during a session, you'll receive an\nerror message requesting that you try again later. Once you're within your quota,\nyou can resume sending requests. The session state, including the session memory,\nare available as long as the session is live.\n\nThis example illustrates how two consecutive requests are processed by\nincluding the tokens from the session memory.\n\n#### Request#1 details\n\n**Duration**: 10 seconds\n\n**Tokens sent (audio)**: 10 seconds x 25 tokens/second = 250 tokens\n\n**Tokens sent (video)**: 10 seconds x 258 tokens/frame per second = 2580 tokens\n\n**Total tokens processed for Request#1**:\n\n- **Tokens sent**: Sum of audio and video tokens sent = 2580+250 = 2830 tokens\n- **Tokens received**: 100 (audio)\n\n#### Request#2 details\n\n**Duration**: 40 seconds\n\n**Tokens sent (audio)**: 40 seconds x 25 tokens/second = 1000 tokens\n\n**Total tokens processed for Request#2**:\n\n- **Tokens sent**: Tokens sent in Request#2 + session memory tokens from Request#1 = 2830 tokens + 1000 tokens = 3830 tokens\n- **Tokens received**: 200 (audio)\n\n#### Calculate the number of tokens processed in the requests\n\nThe number of tokens processed during these requests is calculated, as follows:\n\n- Request#1 processes only the input and output tokens from\n the ongoing request, as there are no additional tokens in the session\n memory.\n\n- Request #2 processes the input and output tokens from\n the ongoing request, but also includes the input tokens from the\n session memory, consisting of the input tokens from the preceding request\n (Request #1) from the session memory. The burndown rate for tokens in the session\n memory is the same as that for standard input tokens\n (1 input session memory token = 1 input token).\n\n If Request#2 took exactly 1 second to process after you sent it,\n your tokens are processed and applied to your Provisioned Throughput quota, as follows:\n - Multiply your inputs by the burndown rates to get the total input tokens:\n\n 2830 x (1 token per session memory token) + 1000 x (1 token per input text token) = 3830 burndown adjusted input tokens per query\n - Multiply your outputs by the burndown rates to get the total output tokens:\n\n 200 x (6 tokens per audio output token) = 1,200 tokens\n - Add these two totals to get the total number of tokens processed:\n\n 3,830 tokens + 1,200 tokens = 5,030 tokens\n\nIf your Provisioned Throughput quota is greater than 5,030 tokens\nper second, then this request can be processed immediately. If it's less, the\ntokens are processed over time at the rate that you've set for your quota.\n\nWhat's next\n-----------\n\n- [Purchase Provisioned Throughput](/vertex-ai/generative-ai/docs/purchase-provisioned-throughput)."]]
