Class RoutingConfig (1.68.0)

RoutingConfig(
    *,
    routing_config: typing.Union[
        vertexai.generative_models._generative_models.GenerationConfig.RoutingConfig.AutoRoutingMode,
        vertexai.generative_models._generative_models.GenerationConfig.RoutingConfig.ManualRoutingMode,
    ]
)

The configuration for model router requests.

The routing config is either one of the two nested classes:

• AutoRoutingMode: Automated routing.
• ManualRoutingMode: Manual routing.

Usage:

• AutoRoutingMode:

  routing_config=generative_models.RoutingConfig(
      routing_config=generative_models.RoutingConfig.AutoRoutingMode(
          model_routing_preference=generative_models.RoutingConfig.AutoRoutingMode.ModelRoutingPreference.BALANCED,
      ),
  )
  

• ManualRoutingMode:

  routing_config=generative_models.RoutingConfig(
      routing_config=generative_models.RoutingConfig.ManutalRoutingMode(
          model_name="gemini-1.5-pro-001",
      ),
  )
  

Classes

AutoRoutingMode

AutoRoutingMode(
    *,
    model_routing_preference: google.cloud.aiplatform_v1beta1.types.content.GenerationConfig.RoutingConfig.AutoRoutingMode.ModelRoutingPreference
)

When automated routing is specified, the routing will be determined by the routing model predicted quality and customer provided model routing preference.

ManualRoutingMode

ManualRoutingMode(*, model_name: str)

When manual routing is set, the specified model will be used directly.