ModelServerInfo(mapping=None, *, ignore_unknown_fields=False, **kwargs)Model server information gives. Valid model server info combinations can be found using GkeInferenceQuickstart.FetchProfiles.
Attributes |
|
|---|---|
| Name | Description |
model |
str
Required. The model. Open-source models follow the Huggingface Hub owner/model_name format. Use
GkeInferenceQuickstart.FetchModels
to find available models.
|
model_server |
str
Required. The model server. Open-source model servers use simplified, lowercase names (e.g., vllm). Use
GkeInferenceQuickstart.FetchModelServers
to find available servers.
|
model_server_version |
str
Optional. The model server version. Use GkeInferenceQuickstart.FetchModelServerVersions to find available versions. If not provided, the latest available version is used. |