Menjalankan inferensi LLM di GPU Cloud Run dengan Hugging Face TGI

Preview — GPU support for Cloud Run services

This feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of the Service Specific Terms. Pre-GA features are available "as is" and might have limited support. For more information, see the launch stage descriptions.

Contoh berikut menunjukkan cara menjalankan layanan backend yang menjalankan toolkit Hugging Face Text Generation Inference (TGI), yang merupakan toolkit untuk men-deploy dan menayangkan Large Language Model (LLM), menggunakan Llama 3.

Lihat seluruh contoh di Men-deploy Llama 3.1 8B dengan DLC TGI di Cloud Run.