使用 Hugging Face TGI 在 Cloud Run GPU 上运行 LLM 推理

Preview — GPU support for Cloud Run services

This feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of the Service Specific Terms. Pre-GA features are available "as is" and might have limited support. For more information, see the launch stage descriptions.

以下示例展示了如何运行后端服务，该服务运行 Hugging Face 文本生成推理 (TGI) 工具包，该工具包用于部署和应用大语言模型 (LLM)。

如需查看完整示例，请参阅在 Cloud Run 上部署包含 TGI DLC 的 Llama 3.1 8B。