Run LLM inference on Cloud Run GPUs with vLLM (services)

Preview — GPU support for Cloud Run services

This feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of the Service Specific Terms. Pre-GA features are available "as is" and might have limited support. For more information, see the launch stage descriptions.

The following codelab shows how to run a backend service that runs vLLM, which is an inference engine for production systems, along with Google's Gemma 2, which is a 2 billion parameters instruction-tuned model.

See the entire codelab at Run LLM inference on Cloud Run GPUs with vLLM.