Google Cloud Aiplatform SDK

class google.cloud.aiplatform.metadata.schema.google.artifact_schema.ExperimentModel(*, framework_name: str, framework_version: str, model_file: str, uri: str, model_class: Optional[str] = None, predict_schemata: Optional[google.cloud.aiplatform.metadata.schema.utils.PredictSchemata] = None, artifact_id: Optional[str] = None, display_name: Optional[str] = None, schema_version: Optional[str] = None, description: Optional[str] = None, metadata: Optional[Dict] = None, state: Optional[google.cloud.aiplatform_v1.types.artifact.Artifact.State] = State.LIVE)

An artifact representing a Vertex Experiment Model.

Instantiates an ExperimentModel that represents a saved ML model.

  • Parameters

    • framework_name (str) – Required. The name of the model’s framework. E.g., ‘sklearn’

    • framework_version (str) – Required. The version of the model’s framework. E.g., ‘1.1.0’

    • model_file (str) – Required. The file name of the model. E.g., ‘model.pkl’

    • uri (str) – Required. The uniform resource identifier of the model artifact directory.

    • model_class (str) – Optional. The class name of the model. E.g., ‘sklearn.linear_model._base.LinearRegression’

    • predict_schemata (PredictSchemata) – Optional. An instance of PredictSchemata which holds instance, parameter and prediction schema uris.

    • artifact_id (str) – Optional. The <resource_id> portion of the Artifact name with the format. This is globally unique in a metadataStore: projects/123/locations/us-central1/metadataStores/<metadata_store_id>/artifacts/<resource_id>.

    • display_name (str) – Optional. The user-defined name of the Artifact.

    • schema_version (str) – Optional. schema_version specifies the version used by the Artifact. If not set, defaults to use the latest version.

    • description (str) – Optional. Describes the purpose of the Artifact to be created.

    • metadata (Dict) – Optional. Contains the metadata information that will be stored in the Artifact.

    • state (google.cloud.gapic.types.Artifact.State) – Optional. The state of this Artifact. This is a property of the Artifact, and does not imply or apture any ongoing process. This property is managed by clients (such as Vertex AI Pipelines), and the system does not prescribe or check the validity of state transitions.

property framework_name(: Optional[str )

The framework name of the saved ML model.

property framework_version(: Optional[str )

The framework version of the saved ML model.

classmethod get(artifact_id: str, *, metadata_store_id: str = 'default', project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Retrieves an existing ExperimentModel artifact given an artifact id.

  • Parameters

    • artifact_id (str) – Required. An artifact id of the ExperimentModel artifact.

    • metadata_store_id (str) – Optional. MetadataStore to retrieve Artifact from. If not set, metadata_store_id is set to “default”. If artifact_id is a fully-qualified resource name, its metadata_store_id overrides this one.

    • project (str) – Optional. Project to retrieve the artifact from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to retrieve the Artifact from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to retrieve this Artifact. Overrides credentials set in aiplatform.init.

  • Returns

    An ExperimentModel class that represents an Artifact resource.

  • Raises

    ValueError – if artifact’s schema title is not ‘google.ExperimentModel’.

get_model_info()

Get the model’s info from an experiment model artifact.

  • Returns

    A dict of model’s info. This includes model’s class name, framework name, framework version, and input example.

load_model()

Retrieves the original ML model from an ExperimentModel.

Example Usage: \ experiment_model = aiplatform.get_experiment_model("my-sklearn-model") sk_model = experiment_model.load_model() pred_y = model.predict(test_X) ``

  • Returns

    The original ML model.

  • Raises

    ValueError – if model type is not supported.

property model_class(: Optional[str )

The class name of the saved ML model.

register_model(*, model_id: Optional[str] = None, parent_model: Optional[str] = None, use_gpu: bool = False, is_default_version: bool = True, version_aliases: Optional[Sequence[str]] = None, version_description: Optional[str] = None, display_name: Optional[str] = None, description: Optional[str] = None, labels: Optional[Dict[str, str]] = None, serving_container_image_uri: Optional[str] = None, serving_container_predict_route: Optional[str] = None, serving_container_health_route: Optional[str] = None, serving_container_command: Optional[Sequence[str]] = None, serving_container_args: Optional[Sequence[str]] = None, serving_container_environment_variables: Optional[Dict[str, str]] = None, serving_container_ports: Optional[Sequence[int]] = None, instance_schema_uri: Optional[str] = None, parameters_schema_uri: Optional[str] = None, prediction_schema_uri: Optional[str] = None, explanation_metadata: Optional[google.cloud.aiplatform_v1.types.explanation_metadata.ExplanationMetadata] = None, explanation_parameters: Optional[google.cloud.aiplatform_v1.types.explanation.ExplanationParameters] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None, encryption_spec_key_name: Optional[str] = None, staging_bucket: Optional[str] = None, sync: Optional[bool] = True, upload_request_timeout: Optional[float] = None)

Register an ExperimentModel to Model Registry and returns a Model representing the registered Model resource.

Example Usage: \ experiment_model = aiplatform.get_experiment_model("my-sklearn-model") registered_model = experiment_model.register_model() registered_model.deploy(endpoint=my_endpoint) ``

  • Parameters

    • model_id (str) – Optional. The ID to use for the registered Model, which will become the final component of the model resource name. This value may be up to 63 characters, and valid characters are [a-z0-9_-]. The first character cannot be a number or hyphen.

    • parent_model (str) – Optional. The resource name or model ID of an existing model that the newly-registered model will be a version of. Only set this field when uploading a new version of an existing model.

    • use_gpu (str) – Optional. Whether or not to use GPUs for the serving container. Only specify this argument when registering a Tensorflow model and ‘serving_container_image_uri’ is not specified.

    • is_default_version (bool) – Optional. When set to True, the newly registered model version will automatically have alias “default” included. Subsequent uses of this model without a version specified will use this “default” version.

      When set to False, the “default” alias will not be moved. Actions targeting the newly-registered model version will need to specifically reference this version by ID or alias.

      New model uploads, i.e. version 1, will always be “default” aliased.

    • version_aliases (Sequence[str]) – Optional. User provided version aliases so that a model version can be referenced via alias instead of auto-generated version ID. A default version alias will be created for the first version of the model.

      The format is [a-z][a-zA-Z0-9-]{0,126}[a-z0-9]

    • version_description (str) – Optional. The description of the model version being uploaded.

    • display_name (str) – Optional. The display name of the Model. The name can be up to 128 characters long and can be consist of any UTF-8 characters.

    • description (str) – Optional. The description of the model.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The labels with user-defined metadata to organize your Models. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information and examples of labels.

    • serving_container_image_uri (str) – Optional. The URI of the Model serving container. A pre-built container [https://cloud.google.com/vertex-ai/docs/predictions/pre-built-containers](https://cloud.google.com/vertex-ai/docs/predictions/pre-built-containers) is automatically chosen based on the model’s framwork. Set this field to override the default pre-built container.

    • serving_container_predict_route (str) – Optional. An HTTP path to send prediction requests to the container, and which must be supported by it. If not specified a default HTTP path will be used by Vertex AI.

    • serving_container_health_route (str) – Optional. An HTTP path to send health check requests to the container, and which must be supported by it. If not specified a standard HTTP path will be used by Vertex AI.

    • serving_container_command (Sequence[str]) – Optional. The command with which the container is run. Not executed within a shell. The Docker image’s ENTRYPOINT is used if this is not provided. Variable references $(VAR_NAME) are expanded using the container’s environment. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not.

    • serving_container_args (Sequence[str]) – Optional. The arguments to the command. The Docker image’s CMD is used if this is not provided. Variable references $(VAR_NAME) are expanded using the container’s environment. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not.

    • serving_container_environment_variables (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The environment variables that are to be present in the container. Should be a dictionary where keys are environment variable names and values are environment variable values for those names.

    • serving_container_ports (Sequence[int]) – Optional. Declaration of ports that are exposed by the container. This field is primarily informational, it gives Vertex AI information about the network connections the container uses. Listing or not a port here has no impact on whether the port is actually exposed, any port listening on the default “0.0.0.0” address inside a container will be accessible from the network.

    • instance_schema_uri (str) – Optional. Points to a YAML file stored on Google Cloud Storage describing the format of a single instance, which are used in PredictRequest.instances, ExplainRequest.instances and BatchPredictionJob.input_config. The schema is defined as an OpenAPI 3.0.2 Schema Object. AutoML Models always have this field populated by AI Platform. Note: The URI given on output will be immutable and probably different, including the URI scheme, than the one given on input. The output URI will point to a location where the user only has a read access.

    • parameters_schema_uri (str) – Optional. Points to a YAML file stored on Google Cloud Storage describing the parameters of prediction and explanation via PredictRequest.parameters, ExplainRequest.parameters and BatchPredictionJob.model_parameters. The schema is defined as an OpenAPI 3.0.2 Schema Object. AutoML Models always have this field populated by AI Platform, if no parameters are supported it is set to an empty string. Note: The URI given on output will be immutable and probably different, including the URI scheme, than the one given on input. The output URI will point to a location where the user only has a read access.

    • prediction_schema_uri (str) – Optional. Points to a YAML file stored on Google Cloud Storage describing the format of a single prediction produced by this Model, which are returned via PredictResponse.predictions, ExplainResponse.explanations, and BatchPredictionJob.output_config. The schema is defined as an OpenAPI 3.0.2 Schema Object. AutoML Models always have this field populated by AI Platform. Note: The URI given on output will be immutable and probably different, including the URI scheme, than the one given on input. The output URI will point to a location where the user only has a read access.

    • explanation_metadata (aiplatform.explain.ExplanationMetadata) – Optional. Metadata describing the Model’s input and output for explanation. explanation_metadata is optional while explanation_parameters must be specified when used. For more details, see Ref docs http://tinyurl.com/1igh60kt

    • explanation_parameters (aiplatform.explain.ExplanationParameters) – Optional. Parameters to configure explaining for Model’s predictions. For more details, see Ref docs http://tinyurl.com/1an4zake

    • project – Optional[str]=None, Project to upload this model to. Overrides project set in aiplatform.init.

    • location – Optional[str]=None, Location to upload this model to. Overrides location set in aiplatform.init.

    • credentials – Optional[auth_credentials.Credentials]=None, Custom credentials to use to upload this model. Overrides credentials set in aiplatform.init.

    • encryption_spec_key_name (Optional[str]) – Optional. The Cloud KMS resource identifier of the customer managed encryption key used to protect the model. Has the form projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key. The key needs to be in the same region as where the compute resource is created.

      If set, this Model and all sub-resources of this Model will be secured by this key.

      Overrides encryption_spec_key_name set in aiplatform.init.

    • staging_bucket (str) – Optional. Bucket to stage local model artifacts. Overrides staging_bucket set in aiplatform.init.

    • sync (bool) – Optional. Whether to execute this method synchronously. If False, this method will unblock and it will be executed in a concurrent Future.

    • upload_request_timeout (float) – Optional. The timeout for the upload request in seconds.

  • Returns

    Instantiated representation of the registered model resource.

  • Return type

    model (aiplatform.Model)

  • Raises

    ValueError – If the model doesn’t have a pre-built container that is suitable for its framework and ‘serving_container_image_uri’ is not set.

class google.cloud.aiplatform.Artifact(artifact_name: str, *, metadata_store_id: str = 'default', project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Bases: google.cloud.aiplatform.metadata.resource._Resource

Metadata Artifact resource for Vertex AI

Retrieves an existing Metadata Artifact given a resource name or ID.

  • Parameters

    • artifact_name (str) – Required. A fully-qualified resource name or resource ID of the Artifact. Example: “projects/123/locations/us-central1/metadataStores/default/artifacts/my-resource”. or “my-resource” when project and location are initialized or passed.

    • metadata_store_id (str) – Optional. MetadataStore to retrieve Artifact from. If not set, metadata_store_id is set to “default”. If artifact_name is a fully-qualified resource, its metadata_store_id overrides this one.

    • project (str) – Optional. Project to retrieve the artifact from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to retrieve the Artifact from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to retrieve this Artifact. Overrides credentials set in aiplatform.init.

client_class()

alias of google.cloud.aiplatform.utils.MetadataClientWithOverride

classmethod create(schema_title: str, *, resource_id: Optional[str] = None, uri: Optional[str] = None, display_name: Optional[str] = None, schema_version: Optional[str] = None, description: Optional[str] = None, metadata: Optional[Dict] = None, state: google.cloud.aiplatform_v1.types.artifact.Artifact.State = State.LIVE, metadata_store_id: Optional[str] = 'default', project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Creates a new Metadata Artifact.

  • Parameters

    • schema_title (str) – Required. schema_title identifies the schema title used by the Artifact.

      Please reference https://cloud.google.com/vertex-ai/docs/ml-metadata/system-schemas.

    • resource_id (str) – Optional. The <resource_id> portion of the Artifact name with the format. This is globally unique in a metadataStore: projects/123/locations/us-central1/metadataStores/<metadata_store_id>/artifacts/<resource_id>.

    • uri (str) – Optional. The uniform resource identifier of the artifact file. May be empty if there is no actual artifact file.

    • display_name (str) – Optional. The user-defined name of the Artifact.

    • schema_version (str) – Optional. schema_version specifies the version used by the Artifact. If not set, defaults to use the latest version.

    • description (str) – Optional. Describes the purpose of the Artifact to be created.

    • metadata (Dict) – Optional. Contains the metadata information that will be stored in the Artifact.

    • state (google.cloud.gapic.types.Artifact.State) – Optional. The state of this Artifact. This is a property of the Artifact, and does not imply or capture any ongoing process. This property is managed by clients (such as Vertex AI Pipelines), and the system does not prescribe or check the validity of state transitions.

    • metadata_store_id (str) – Optional. The <metadata_store_id> portion of the resource name with the format: projects/123/locations/us-central1/metadataStores/<metadata_store_id>/artifacts/<resource_id> If not provided, the MetadataStore’s ID will be set to “default”.

    • project (str) – Optional. Project used to create this Artifact. Overrides project set in aiplatform.init.

    • location (str) – Optional. Location used to create this Artifact. Overrides location set in aiplatform.init.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials used to create this Artifact. Overrides credentials set in aiplatform.init.

  • Returns

    Instantiated representation of the managed Metadata Artifact.

  • Return type

    Artifact

property create_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was created.

delete(sync: bool = True)

Deletes this Vertex AI resource. WARNING: This deletion is permanent.

  • Parameters

    sync (bool) – Whether to execute this deletion synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

property display_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Display name of this resource.

property encryption_spec(: Optional[[google.cloud.aiplatform_v1.types.encryption_spec.EncryptionSpec](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.EncryptionSpec)_ )

Customer-managed encryption key options for this Vertex AI resource.

If this is set, then all resources created by this Vertex AI resource will be encrypted with the provided encryption key.

property gca_resource(: [proto.message.Message](https://proto-plus-python.readthedocs.io/en/latest/reference/message.html#proto.message.Message )

The underlying resource proto representation.

classmethod get(resource_id: str, metadata_store_id: str = 'default', project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Retrieves a Metadata resource.

  • Parameters

    • resource_id (str) – Required. The <resource_id> portion of the resource name with the format: projects/123/locations/us-central1/metadataStores/<metadata_store_id>/<resource_noun>/<resource_id>.

    • metadata_store_id (str) – The <metadata_store_id> portion of the resource name with the format: projects/123/locations/us-central1/metadataStores/<metadata_store_id>/<resource_noun>/<resource_id> If not provided, the MetadataStore’s ID will be set to “default”.

    • project (str) – Project used to retrieve or create this resource. Overrides project set in aiplatform.init.

    • location (str) – Location used to retrieve or create this resource. Overrides location set in aiplatform.init.

    • credentials (auth_credentials.Credentials) – Custom credentials used to retrieve or create this resource. Overrides credentials set in aiplatform.init.

  • Returns

    Instantiated representation of the managed Metadata resource or None if no resource was found.

  • Return type

    resource (_Resource)

classmethod get_or_create(resource_id: str, schema_title: str, display_name: Optional[str] = None, schema_version: Optional[str] = None, description: Optional[str] = None, metadata: Optional[Dict] = None, metadata_store_id: str = 'default', project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Retrieves or Creates (if it does not exist) a Metadata resource.

  • Parameters

    • resource_id (str) – Required. The <resource_id> portion of the resource name with the format: projects/123/locations/us-central1/metadataStores/<metadata_store_id>/<resource_noun>/<resource_id>.

    • schema_title (str) – Required. schema_title identifies the schema title used by the resource.

    • display_name (str) – Optional. The user-defined name of the resource.

    • schema_version (str) – Optional. schema_version specifies the version used by the resource. If not set, defaults to use the latest version.

    • description (str) – Optional. Describes the purpose of the resource to be created.

    • metadata (Dict) – Optional. Contains the metadata information that will be stored in the resource.

    • metadata_store_id (str) – The <metadata_store_id> portion of the resource name with the format: projects/123/locations/us-central1/metadataStores/<metadata_store_id>/<resource_noun>/<resource_id> If not provided, the MetadataStore’s ID will be set to “default”.

    • project (str) – Project used to retrieve or create this resource. Overrides project set in aiplatform.init.

    • location (str) – Location used to retrieve or create this resource. Overrides location set in aiplatform.init.

    • credentials (auth_credentials.Credentials) – Custom credentials used to retrieve or create this resource. Overrides credentials set in aiplatform.init.

  • Returns

    Instantiated representation of the managed Metadata resource.

  • Return type

    resource (_Resource)

classmethod get_with_uri(uri: str, *, metadata_store_id: Optional[str] = 'default', project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Get an Artifact by it’s uri.

If more than one Artifact with this uri is in the metadata store then the Artifact with the latest create_time is returned.

  • Parameters

    • uri (str) – Required. Uri of the Artifact to retrieve.

    • metadata_store_id (str) – Optional. MetadataStore to retrieve Artifact from. If not set, metadata_store_id is set to “default”. If artifact_name is a fully-qualified resource, its metadata_store_id overrides this one.

    • project (str) – Optional. Project to retrieve the artifact from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to retrieve the Artifact from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to retrieve this Artifact. Overrides credentials set in aiplatform.init.

  • Returns

    Artifact with given uri.

  • Return type

    Artifact

  • Raises

    ValueError – If no Artifact exists with the provided uri.

property labels(: Dict[str, str )

User-defined labels containing metadata about this resource.

Read more about labels at https://goo.gl/xmQnxf

property lineage_console_uri(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Cloud console uri to view this Artifact Lineage.

classmethod list(filter: Optional[str] = None, metadata_store_id: str = 'default', project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None, order_by: Optional[str] = None)

List resources that match the list filter in target metadataStore.

  • Parameters

    • filter (str) – Optional. A query to filter available resources for matching results.

    • metadata_store_id (str) – The <metadata_store_id> portion of the resource name with the format: projects/123/locations/us-central1/metadataStores/<metadata_store_id>/<resource_noun>/<resource_id> If not provided, the MetadataStore’s ID will be set to “default”.

    • project (str) – Project used to create this resource. Overrides project set in aiplatform.init.

    • location (str) – Location used to create this resource. Overrides location set in aiplatform.init.

    • credentials (auth_credentials.Credentials) – Custom credentials used to create this resource. Overrides credentials set in aiplatform.init.

    • order_by (str) – Optional. How the list of messages is ordered. Specify the values to order by and an ordering operation. The default sorting order is ascending. To specify descending order for a field, users append a ” desc” suffix; for example: “foo desc, bar”. Subfields are specified with a . character, such as foo.bar. see https://google.aip.dev/132#ordering for more details.

  • Returns

    a list of managed Metadata resource.

  • Return type

    resources (sequence[_Resource])

property name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Name of this resource.

property resource_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Full qualified resource name.

property state(: Optional[[google.cloud.aiplatform_v1.types.artifact.Artifact.State](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.Artifact.State)_ )

The State for this Artifact.

sync_resource()

Syncs local resource with the resource in metadata store.

to_dict()

Returns the resource proto as a dictionary.

update(metadata: Optional[Dict] = None, description: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Updates an existing Metadata resource with new metadata.

  • Parameters

    • metadata (Dict) – Optional. metadata contains the updated metadata information.

    • description (str) – Optional. Description describes the resource to be updated.

    • credentials (auth_credentials.Credentials) – Custom credentials to use to update this resource. Overrides credentials set in aiplatform.init.

property update_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was last updated.

property uri(: Optional[str )

Uri for this Artifact.

wait()

Helper method that blocks until all futures are complete.

class google.cloud.aiplatform.AutoMLForecastingTrainingJob(display_name: Optional[str] = None, optimization_objective: Optional[str] = None, column_specs: Optional[Dict[str, str]] = None, column_transformations: Optional[List[Dict[str, Dict[str, str]]]] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None, labels: Optional[Dict[str, str]] = None, training_encryption_spec_key_name: Optional[str] = None, model_encryption_spec_key_name: Optional[str] = None)

Bases: google.cloud.aiplatform.training_jobs._ForecastingTrainingJob

Class to train AutoML forecasting models.

Constructs a Forecasting Training Job.

  • Parameters

    • display_name (str) – Optional. The user-defined name of this TrainingPipeline.

    • optimization_objective (str) – Optional. Objective function the model is to be optimized towards. The training process creates a Model that optimizes the value of the objective function over the validation set. The supported optimization objectives: “minimize-rmse” (default) - Minimize root-mean-squared error (RMSE). “minimize-mae” - Minimize mean-absolute error (MAE). “minimize-rmsle” - Minimize root-mean-squared log error (RMSLE). “minimize-rmspe” - Minimize root-mean-squared percentage error (RMSPE). “minimize-wape-mae” - Minimize the combination of weighted absolute percentage error (WAPE)

      and mean-absolute-error (MAE).

      ”minimize-quantile-loss” - Minimize the quantile loss at the defined quantiles.

      (Set this objective to build quantile forecasts.)

    • column_specs (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. Alternative to column_transformations where the keys of the dict are column names and their respective values are one of AutoMLTabularTrainingJob.column_data_types. When creating transformation for BigQuery Struct column, the column should be flattened using “.” as the delimiter. Only columns with no child should have a transformation. If an input column has no transformations on it, such a column is ignored by the training, except for the targetColumn, which should have no transformations defined on. Only one of column_transformations or column_specs should be passed.

    • column_transformations (List[Dict[str, **Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]]]*) – Optional. Transformations to apply to the input columns (i.e. columns other than the targetColumn). Each transformation may produce multiple result values from the column’s value, and all are used for training. When creating transformation for BigQuery Struct column, the column should be flattened using “.” as the delimiter. Only columns with no child should have a transformation. If an input column has no transformations on it, such a column is ignored by the training, except for the targetColumn, which should have no transformations defined on. Only one of column_transformations or column_specs should be passed. Consider using column_specs as column_transformations will be deprecated eventually.

    • project (str) – Optional. Project to run training in. Overrides project set in aiplatform.init.

    • location (str) – Optional. Location to run training in. Overrides location set in aiplatform.init.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to run call training service. Overrides credentials set in aiplatform.init.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The labels with user-defined metadata to organize TrainingPipelines. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information and examples of labels.

    • training_encryption_spec_key_name (Optional[str]) – Optional. The Cloud KMS resource identifier of the customer managed encryption key used to protect the training pipeline. Has the form: projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key. The key needs to be in the same region as where the compute resource is created. If set, this TrainingPipeline will be secured by this key. Note: Model trained by this TrainingPipeline is also secured by this key if model_to_upload is not set separately. Overrides encryption_spec_key_name set in aiplatform.init.

    • model_encryption_spec_key_name (Optional[str]) – Optional. The Cloud KMS resource identifier of the customer managed encryption key used to protect the model. Has the form: projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key. The key needs to be in the same region as where the compute resource is created. If set, the trained Model will be secured by this key. Overrides encryption_spec_key_name set in aiplatform.init.

  • Raises

    ValueError – If both column_transformations and column_specs were provided.

cancel()

Starts asynchronous cancellation on the TrainingJob. The server makes a best effort to cancel the job, but success is not guaranteed. On successful cancellation, the TrainingJob is not deleted; instead it becomes a job with state set to CANCELLED.

  • Raises

    RuntimeError – If this TrainingJob has not started running.

client_class()

alias of google.cloud.aiplatform.utils.PipelineClientWithOverride

property create_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was created.

delete(sync: bool = True)

Deletes this Vertex AI resource. WARNING: This deletion is permanent.

  • Parameters

    sync (bool) – Whether to execute this deletion synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

property display_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Display name of this resource.

done()

Method indicating whether a job has completed.

  • Returns

    True if the job has completed.

property encryption_spec(: Optional[[google.cloud.aiplatform_v1.types.encryption_spec.EncryptionSpec](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.EncryptionSpec)_ )

Customer-managed encryption key options for this Vertex AI resource.

If this is set, then all resources created by this Vertex AI resource will be encrypted with the provided encryption key.

property end_time(: Optional[datetime.datetime )

Time when the TrainingJob resource entered the PIPELINE_STATE_SUCCEEDED, PIPELINE_STATE_FAILED, PIPELINE_STATE_CANCELLED state.

property error(: Optional[google.rpc.status_pb2.Status )

Detailed error info for this TrainingJob resource. Only populated when the TrainingJob’s state is PIPELINE_STATE_FAILED or PIPELINE_STATE_CANCELLED.

property evaluated_data_items_bigquery_uri(: Optional[str )

BigQuery location of exported evaluated examples from the Training Job :returns:

BigQuery uri for the exported evaluated examples if the export

feature is enabled for training.

None: If the export feature was not enabled for training.

  • Return type

    str

property gca_resource(: [proto.message.Message](https://proto-plus-python.readthedocs.io/en/latest/reference/message.html#proto.message.Message )

The underlying resource proto representation.

classmethod get(resource_name: str, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Get Training Job for the given resource_name.

  • Parameters

    • resource_name (str) – Required. A fully-qualified resource name or ID.

    • project (str) – Optional project to retrieve training job from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional location to retrieve training job from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Custom credentials to use to upload this model. Overrides credentials set in aiplatform.init.

  • Raises

    ValueError – If the retrieved training job’s training task definition doesn’t match the custom training task definition.

  • Returns

    A Vertex AI Training Job

get_model(sync=True)

Vertex AI Model produced by this training, if one was produced.

  • Parameters

    sync (bool) – Whether to execute this method synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

  • Returns

    Vertex AI Model produced by this training

  • Return type

    model

  • Raises

    RuntimeError – If training failed or if a model was not produced by this training.

property has_failed(: [bool](https://python.readthedocs.io/en/latest/library/functions.html#bool )

Returns True if training has failed.

False otherwise.

property labels(: Dict[str, str )

User-defined labels containing metadata about this resource.

Read more about labels at https://goo.gl/xmQnxf

classmethod list(filter: Optional[str] = None, order_by: Optional[str] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

List all instances of this TrainingJob resource.

Example Usage:

aiplatform.CustomTrainingJob.list(

filter=’display_name=”experiment_a27”’,
order_by=’create_time desc’

)

  • Parameters

    • filter (str) – Optional. An expression for filtering the results of the request. For field names both snake_case and camelCase are supported.

    • order_by (str) – Optional. A comma-separated list of fields to order by, sorted in ascending order. Use “desc” after a field name for descending. Supported fields: display_name, create_time, update_time

    • project (str) – Optional. Project to retrieve list from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to retrieve list from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to retrieve list. Overrides credentials set in aiplatform.init.

  • Returns

    List[VertexAiResourceNoun] - A list of TrainingJob resource objects

property name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Name of this resource.

property resource_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Full qualified resource name.

run(dataset: google.cloud.aiplatform.datasets.time_series_dataset.TimeSeriesDataset, target_column: str, time_column: str, time_series_identifier_column: str, unavailable_at_forecast_columns: List[str], available_at_forecast_columns: List[str], forecast_horizon: int, data_granularity_unit: str, data_granularity_count: int, training_fraction_split: Optional[float] = None, validation_fraction_split: Optional[float] = None, test_fraction_split: Optional[float] = None, predefined_split_column_name: Optional[str] = None, timestamp_split_column_name: Optional[str] = None, weight_column: Optional[str] = None, time_series_attribute_columns: Optional[List[str]] = None, context_window: Optional[int] = None, export_evaluated_data_items: bool = False, export_evaluated_data_items_bigquery_destination_uri: Optional[str] = None, export_evaluated_data_items_override_destination: bool = False, quantiles: Optional[List[float]] = None, validation_options: Optional[str] = None, budget_milli_node_hours: int = 1000, model_display_name: Optional[str] = None, model_labels: Optional[Dict[str, str]] = None, model_id: Optional[str] = None, parent_model: Optional[str] = None, is_default_version: Optional[bool] = True, model_version_aliases: Optional[Sequence[str]] = None, model_version_description: Optional[str] = None, additional_experiments: Optional[List[str]] = None, hierarchy_group_columns: Optional[List[str]] = None, hierarchy_group_total_weight: Optional[float] = None, hierarchy_temporal_total_weight: Optional[float] = None, hierarchy_group_temporal_total_weight: Optional[float] = None, window_column: Optional[str] = None, window_stride_length: Optional[int] = None, window_max_count: Optional[int] = None, holiday_regions: Optional[List[str]] = None, sync: bool = True, create_request_timeout: Optional[float] = None, enable_probabilistic_inference: bool = False)

Runs the training job and returns a model.

If training on a Vertex AI dataset, you can use one of the following split configurations:

Data fraction splits:
Any of `training_fraction_split`, `validation_fraction_split` and
`test_fraction_split` may optionally be provided, they must sum to up to 1. If
the provided ones sum to less than 1, the remainder is assigned to sets as
decided by Vertex AI. If none of the fractions are set, by default roughly 80%
of data will be used for training, 10% for validation, and 10% for test.

Predefined splits:
Assigns input data to training, validation, and test sets based on the value of a provided key.
If using predefined splits, `predefined_split_column_name` must be provided.
Supported only for tabular Datasets.

Timestamp splits:
Assigns input data to training, validation, and test sets
based on a provided timestamps. The youngest data pieces are
assigned to training set, next to validation set, and the oldest
to the test set.
Supported only for tabular Datasets.

  • Parameters

    • dataset (datasets.TimeSeriesDataset) – Required. The dataset within the same Project from which data will be used to train the Model. The Dataset must use schema compatible with Model being trained, and what is compatible should be described in the used TrainingPipeline’s [training_task_definition] [google.cloud.aiplatform.v1beta1.TrainingPipeline.training_task_definition]. For time series Datasets, all their data is exported to training, to pick and choose from.

    • target_column (str) – Required. Name of the column that the Model is to predict values for. This column must be unavailable at forecast.

    • time_column (str) – Required. Name of the column that identifies time order in the time series. This column must be available at forecast.

    • time_series_identifier_column (str) – Required. Name of the column that identifies the time series.

    • unavailable_at_forecast_columns (List[str]) – Required. Column names of columns that are unavailable at forecast. Each column contains information for the given entity (identified by the [time_series_identifier_column]) that is unknown before the forecast (e.g. population of a city in a given year, or weather on a given day).

    • available_at_forecast_columns (List[str]) – Required. Column names of columns that are available at forecast. Each column contains information for the given entity (identified by the [time_series_identifier_column]) that is known at forecast.

    • forecast_horizon – (int): Required. The amount of time into the future for which forecasted values for the target are returned. Expressed in number of units defined by the [data_granularity_unit] and [data_granularity_count] field. Inclusive.

    • data_granularity_unit (str) – Required. The data granularity unit. Accepted values are minute, hour, day, week, month, year.

    • data_granularity_count (int) – Required. The number of data granularity units between data points in the training data. If [data_granularity_unit] is minute, can be 1, 5, 10, 15, or 30. For all other values of [data_granularity_unit], must be 1.

    • predefined_split_column_name (str) – Optional. The key is a name of one of the Dataset’s data columns. The value of the key (either the label’s value or value in the column) must be one of {TRAIN, VALIDATE, TEST}, and it defines to which set the given piece of data is assigned. If for a piece of data the key is not present or has an invalid value, that piece is ignored by the pipeline.

      Supported only for tabular and time series Datasets.

    • timestamp_split_column_name (str) – Optional. The key is a name of one of the Dataset’s data columns. The value of the key values of the key (the values in the column) must be in RFC 3339 date-time format, where time-offset = “Z” (e.g. 1985-04-12T23:20:50.52Z). If for a piece of data the key is not present or has an invalid value, that piece is ignored by the pipeline. Supported only for tabular and time series Datasets. This parameter must be used with training_fraction_split, validation_fraction_split, and test_fraction_split.

    • weight_column (str) – Optional. Name of the column that should be used as the weight column. Higher values in this column give more importance to the row during Model training. The column must have numeric values between 0 and 10000 inclusively, and 0 value means that the row is ignored. If the weight column field is not set, then all rows are assumed to have equal weight of 1.

    • time_series_attribute_columns (List[str]) – Optional. Column names that should be used as attribute columns. Each column is constant within a time series.

    • context_window (int) – Optional. The amount of time into the past training and prediction data is used for model training and prediction respectively. Expressed in number of units defined by the [data_granularity_unit] and [data_granularity_count] fields. When not provided uses the default value of 0 which means the model sets each series context window to be 0 (also known as “cold start”). Inclusive.

    • export_evaluated_data_items (bool) – Whether to export the test set predictions to a BigQuery table. If False, then the export is not performed.

    • export_evaluated_data_items_bigquery_destination_uri (string) – Optional. URI of desired destination BigQuery table for exported test set predictions.

      Expected format: bq://<project_id>:<dataset_id>:<table>

      If not specified, then results are exported to the following auto-created BigQuery table: <project_id>:export_evaluated_examples_<model_name>_<yyyy_MM_dd'T'HH_mm_ss_SSS'Z'>.evaluated_examples

      Applies only if [export_evaluated_data_items] is True.

    • export_evaluated_data_items_override_destination (bool) – Whether to override the contents of [export_evaluated_data_items_bigquery_destination_uri], if the table exists, for exported test set predictions. If False, and the table exists, then the training job will fail.

      Applies only if [export_evaluated_data_items] is True and [export_evaluated_data_items_bigquery_destination_uri] is specified.

    • quantiles (List[float]) – Quantiles to use for the minimize-quantile-loss [AutoMLForecastingTrainingJob.optimization_objective]. This argument is required in this case.

      Accepts up to 5 quantiles in the form of a double from 0 to 1, exclusive. Each quantile must be unique.

    • validation_options (str) – Validation options for the data validation component. The available options are: “fail-pipeline” - (default), will validate against the validation and fail the pipeline

      if it fails.

      ”ignore-validation” - ignore the results of the validation and continue the pipeline

    • budget_milli_node_hours (int) – Optional. The train budget of creating this Model, expressed in milli node hours i.e. 1,000 value in this field means 1 node hour. The training cost of the model will not exceed this budget. The final cost will be attempted to be close to the budget, though may end up being (even) noticeably smaller - at the backend’s discretion. This especially may happen when further model training ceases to provide any improvements. If the budget is set to a value known to be insufficient to train a Model for the given training set, the training won’t be attempted and will error. The minimum value is 1000 and the maximum is 72000.

    • model_display_name (str) – Optional. If the script produces a managed Vertex AI Model. The display name of the Model. The name can be up to 128 characters long and can be consist of any UTF-8 characters.

      If not provided upon creation, the job’s display_name is used.

    • model_labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The labels with user-defined metadata to organize your Models. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information and examples of labels.

    • model_id (str) – Optional. The ID to use for the Model produced by this job, which will become the final component of the model resource name. This value may be up to 63 characters, and valid characters are [a-z0-9_-]. The first character cannot be a number or hyphen.

    • parent_model (str) – Optional. The resource name or model ID of an existing model. The new model uploaded by this job will be a version of parent_model.

      Only set this field when training a new version of an existing model.

    • is_default_version (bool) – Optional. When set to True, the newly uploaded model version will automatically have alias “default” included. Subsequent uses of the model produced by this job without a version specified will use this “default” version.

      When set to False, the “default” alias will not be moved. Actions targeting the model version produced by this job will need to specifically reference this version by ID or alias.

      New model uploads, i.e. version 1, will always be “default” aliased.

    • model_version_aliases (Sequence[str]) – Optional. User provided version aliases so that the model version uploaded by this job can be referenced via alias instead of auto-generated version ID. A default version alias will be created for the first version of the model.

      The format is [a-z][a-zA-Z0-9-]{0,126}[a-z0-9]

    • model_version_description (str) – Optional. The description of the model version being uploaded by this job.

    • additional_experiments (List[str]) – Optional. Additional experiment flags for the time series forcasting training.

    • create_request_timeout (float) – Optional. The timeout for the create request in seconds.

    • hierarchy_group_columns (List[str]) – Optional. A list of time series attribute column names that define the time series hierarchy. Only one level of hierarchy is supported, ex. region for a hierarchy of stores or department for a hierarchy of products. If multiple columns are specified, time series will be grouped by their combined values, ex. (blue, large) for color and size, up to 5 columns are accepted. If no group columns are specified, all time series are considered to be part of the same group.

    • hierarchy_group_total_weight (float) – Optional. The weight of the loss for predictions aggregated over time series in the same hierarchy group.

    • hierarchy_temporal_total_weight (float) – Optional. The weight of the loss for predictions aggregated over the horizon for a single time series.

    • hierarchy_group_temporal_total_weight (float) – Optional. The weight of the loss for predictions aggregated over both the horizon and time series in the same hierarchy group.

    • window_column (str) – Optional. Name of the column that should be used to filter input rows. The column should contain either booleans or string booleans; if the value of the row is True, generate a sliding window from that row.

    • window_stride_length (int) – Optional. Step length used to generate input examples. Every window_stride_length rows will be used to generate a sliding window.

    • window_max_count (int) – Optional. Number of rows that should be used to generate input examples. If the total row count is larger than this number, the input data will be randomly sampled to hit the count.

    • holiday_regions (List[str]) – Optional. The geographical regions to use when creating holiday features. This option is only allowed when data_granularity_unit is day. Acceptable values can come from any of the following levels:

      Top level: GLOBAL Second level: continental regions

      NA: North America JAPAC: Japan and Asia Pacific EMEA: Europe, the Middle East and Africa LAC: Latin America and the Caribbean

      Third level: countries from ISO 3166-1 Country codes.

    • sync (bool) – Whether to execute this method synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

    • enable_probabilistic_inference (bool) – If probabilistic inference is enabled, the model will fit a distribution that captures the uncertainty of a prediction. At inference time, the predictive distribution is used to make a point prediction that minimizes the optimization objective. For example, the mean of a predictive distribution is the point prediction that minimizes RMSE loss. If quantiles are specified, then the quantiles of the distribution are also returned. The optimization objective cannot be minimize-quantile-loss.

  • Returns

    The trained Vertex AI Model resource or None if training did not

      produce a Vertex AI Model.

  • Return type

    model

  • Raises

    RuntimeError – If Training job has already been run or is waiting to run.

property start_time(: Optional[datetime.datetime )

Time when the TrainingJob entered the PIPELINE_STATE_RUNNING for the first time.

property state(: Optional[[google.cloud.aiplatform_v1.types.pipeline_state.PipelineState](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.PipelineState)_ )

Current training state.

to_dict()

Returns the resource proto as a dictionary.

property update_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was last updated.

wait()

Helper method that blocks until all futures are complete.

wait_for_resource_creation()

Waits until resource has been created.

class google.cloud.aiplatform.AutoMLImageTrainingJob(display_name: Optional[str] = None, prediction_type: str = 'classification', multi_label: bool = False, model_type: str = 'CLOUD', base_model: Optional[google.cloud.aiplatform.models.Model] = None, incremental_train_base_model: Optional[google.cloud.aiplatform.models.Model] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None, labels: Optional[Dict[str, str]] = None, training_encryption_spec_key_name: Optional[str] = None, model_encryption_spec_key_name: Optional[str] = None, checkpoint_name: Optional[str] = None, trainer_config: Optional[Dict[str, str]] = None, metric_spec: Optional[Dict[str, str]] = None, parameter_spec: Optional[Dict[str, Union[google.cloud.aiplatform.hyperparameter_tuning.DoubleParameterSpec, google.cloud.aiplatform.hyperparameter_tuning.IntegerParameterSpec, google.cloud.aiplatform.hyperparameter_tuning.CategoricalParameterSpec, google.cloud.aiplatform.hyperparameter_tuning.DiscreteParameterSpec]]] = None, search_algorithm: Optional[str] = None, measurement_selection: Optional[str] = None)

Bases: google.cloud.aiplatform.training_jobs._TrainingJob

Constructs a AutoML Image Training Job.

  • Parameters

    • display_name (str) – Optional. The user-defined name of this TrainingPipeline.

    • prediction_type (str) – The type of prediction the Model is to produce, one of: “classification” - Predict one out of multiple target values is picked for each row. “object_detection” - Predict a value based on its relation to other values. This type is available only to columns that contain semantically numeric values, i.e. integers or floating point number, even if stored as e.g. strings.

    • multi_label – bool = False Required. Default is False. If false, a single-label (multi-class) Model will be trained (i.e. assuming that for each image just up to one annotation may be applicable). If true, a multi-label Model will be trained (i.e. assuming that for each image multiple annotations may be applicable). This is only applicable for the “classification” prediction_type and will be ignored otherwise.

    • model_type – str = “CLOUD”

      Required. One of the following:

      ”CLOUD” - Default for Image Classification. A Model best tailored to be used within Google Cloud, and which cannot be exported.

      ”CLOUD_1” - A model type best tailored to be used within Google Cloud, which cannot be exported externally. Compared to the CLOUD model above, it is expected to have higher prediction accuracy.

      ”CLOUD_HIGH_ACCURACY_1” - Default for Image Object Detection. A model best tailored to be used within Google Cloud, and which cannot be exported. Expected to have a higher latency, but should also have a higher prediction quality than other cloud models.

      ”CLOUD_LOW_LATENCY_1” - A model best tailored to be used within Google Cloud, and which cannot be exported. Expected to have a low latency, but may have lower prediction quality than other cloud models.

      ”MOBILE_TF_LOW_LATENCY_1” - A model that, in addition to being available within Google Cloud, can also be exported as TensorFlow or Core ML model and used on a mobile or edge device afterwards. Expected to have low latency, but may have lower prediction quality than other mobile models.

      ”MOBILE_TF_VERSATILE_1” - A model that, in addition to being available within Google Cloud, can also be exported as TensorFlow or Core ML model and used on a mobile or edge device with afterwards.

      ”MOBILE_TF_HIGH_ACCURACY_1” - A model that, in addition to being available within Google Cloud, can also be exported as TensorFlow or Core ML model and used on a mobile or edge device afterwards. Expected to have a higher latency, but should also have a higher prediction quality than other mobile models.

      ”EFFICIENTNET” - A model that, available in Vertex Model Garden image classification training with customizable hyperparameters. Best tailored to be used within Google Cloud, and cannot be exported externally.

      ”VIT” - A model that, available in Vertex Model Garden image classification training with customizable hyperparameters. Best tailored to be used within Google Cloud, and cannot be exported externally.

      ”MAXVIT” - A model that, available in Vertex Model Garden image classification training with customizable hyperparameters. Best tailored to be used within Google Cloud, and cannot be exported externally.

      ”COCA” - A model that, available in Vertex Model Garden image classification training with customizable hyperparameters. Best tailored to be used within Google Cloud, and cannot be exported externally.

      ”SPINENET” - A model that, available in Vertex Model Garden image object detection training with customizable hyperparameters. Best tailored to be used within Google Cloud, and cannot be exported externally.

      ”YOLO” - A model that, available in Vertex Model Garden image object detection training with customizable hyperparameters. Best tailored to be used within Google Cloud, and cannot be exported externally.

    • base_model – Optional[models.Model] = None Optional. Only permitted for Image Classification models. If it is specified, the new model will be trained based on the base model. Otherwise, the new model will be trained from scratch. The base model must be in the same Project and Location as the new Model to train, and have the same model_type.

    • incremental_train_base_model – Optional[models.Model] = None Optional for both Image Classification and Object detection models, to incrementally train a new model using an existing model as the starting point, with a reduced training time. If not specified, the new model will be trained from scratch. The base model must be in the same Project and Location as the new Model to train, and have the same prediction_type and model_type.

    • project (str) – Optional. Project to run training in. Overrides project set in aiplatform.init.

    • location (str) – Optional. Location to run training in. Overrides location set in aiplatform.init.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to run call training service. Overrides credentials set in aiplatform.init.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The labels with user-defined metadata to organize TrainingPipelines. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information and examples of labels.

    • training_encryption_spec_key_name (Optional[str]) – Optional. The Cloud KMS resource identifier of the customer managed encryption key used to protect the training pipeline. Has the

      form:

      `projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key`.
The key needs to be in the same region as where the compute
resource is created.  If set, this TrainingPipeline will be
secured by this key.

      Note: Model trained by this TrainingPipeline is also secured by this

      key if `model_to_upload` is not set separately.  Overrides
encryption_spec_key_name set in aiplatform.init.

    • model_encryption_spec_key_name (Optional[str]) – Optional. The Cloud KMS resource identifier of the customer managed encryption key used to protect the model. Has the

      form:

      `projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key`.
The key needs to be in the same region as where the compute
resource is created.  If set, the trained Model will be secured by
this key.  Overrides encryption_spec_key_name set in
aiplatform.init.

    • checkpoint_name – Optional[str] = None, Optional. The field is reserved for Model Garden model training, based on the provided pre-trained model checkpoint. Only necessary for 

      `

      model_type`==EFFICIENTNET, VIT, COCA, SPINENET, YOLO.

    • trainer_config – Optional[Dict[str, str]] = None, Optional. The field is usually used together with the Model Garden model training, when passing the customized configs for the trainer.

      Example with all supported parameters:

      trainer_config = {

      ‘global_batch_size’: ‘8’, ‘learning_rate’: ‘0.001’, ‘optimizer_type’: ‘sgd’, ‘optimizer_momentum’: ‘0.9’, ‘train_steps’: ‘10000’, ‘accelerator_count’: ‘1’, ‘anchor_size’: ‘8’, – IOD only

      } trainer_config is only required for Model Garden models when

```python
`
```

model_type\`==EFFICIENTNET, VIT, COCA, SPINENET, YOLO.



* **metric_spec** – Dict[str, str] Required. Dictionary representing metrics to
optimize. The dictionary key is the metric_id, which is reported by
your training job, with possible values be (‘loss’, ‘accuracy’) and the
dictionary value is the optimization goal of the metric(‘minimize’ or ‘maximize’).

example:  metric_spec = {‘loss’: ‘minimize’, ‘accuracy’: ‘maximize’}
metric_spec is only required for Model Garden models when


```python
`
```

model_type\`==EFFICIENTNET, VIT, COCA, SPINENET, YOLO.



* **parameter_spec** (*Dict**[*[*str*](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)*, **hpt._ParameterSpec**]*) – Required. Dictionary
representing parameters to optimize. The dictionary key is the
metric_id, which is passed into your training job as a command line
key word argument, and the dictionary value is the parameter
specification of the metric.

from google.cloud.aiplatform import hpt as hpt

parameter_spec = {

    ‘learning_rate’: hpt.DoubleParameterSpec(min=1e-7, max=1, scale=’linear’),

}

Supported parameter specifications can be found in aiplatform.hyperparameter_tuning.
These parameter specification are currently supported: Union[DoubleParameterSpec,
IntegerParameterSpec, CategoricalParameterSpec, DiscreteParameterSpec]
parameter_spec is only required for Model Garden models when


```python
`
```

model_type\`==EFFICIENTNET, VIT, COCA, SPINENET, YOLO.



* **search_algorithm** ([*str*](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)) – The search algorithm specified for the Study.
Accepts one of the following:

> None - If you do not specify an algorithm, your job uses the default
> Vertex AI algorithm. The default algorithm applies Bayesian optimization
> to arrive at the optimal solution with a more effective search over the
> parameter space.

> ’grid’ - A simple grid search within the feasible space. This option is
> particularly useful if you want to specify a quantity of trials that is greater
> than the number of points in the feasible space. In such cases, if you do
> not specify a grid search, the Vertex AI default algorithm may generate
> duplicate suggestions. To use grid search, all parameter specs must be of
> type IntegerParameterSpec, CategoricalParameterSpec, or DiscreteParameterSpec.

> ’random’ - A simple random search within the feasible space.

search_algorithm is only required for Model Garden models when


```python
`
```

model_type\`==EFFICIENTNET, VIT, COCA, SPINENET, YOLO.



* **measurement_selection** ([*str*](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)) – This indicates which measurement to use
if/when the service automatically selects the final measurement from
previously reported intermediate measurements.

> Accepts: ‘best’, ‘last’  Choose this based on two considerations: A)

>     Do you expect your measurements to monotonically improve? If so,
>     choose ‘last’. On the other hand, if you’re in a situation where
>     your system can “over-train” and you expect the performance to get
>     better for a while but then start declining, choose ‘best’. B) Are
>     your measurements significantly noisy and/or irreproducible? If
>     so, ‘best’ will tend to be over-optimistic, and it may be better
>     to choose ‘last’. If both or neither of (A) and (B) apply, it
>     doesn’t matter which selection type is chosen.

measurement_selection is only required for Model Garden models when


```python
`
```

model_type\`==EFFICIENTNET, VIT, COCA, SPINENET, YOLO.

  • Raises

    ValueError – When an invalid prediction_type or model_type is provided.

cancel()

Starts asynchronous cancellation on the TrainingJob. The server makes a best effort to cancel the job, but success is not guaranteed. On successful cancellation, the TrainingJob is not deleted; instead it becomes a job with state set to CANCELLED.

  • Raises

    RuntimeError – If this TrainingJob has not started running.

client_class()

alias of google.cloud.aiplatform.utils.PipelineClientWithOverride

property create_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was created.

delete(sync: bool = True)

Deletes this Vertex AI resource. WARNING: This deletion is permanent.

  • Parameters

    sync (bool) – Whether to execute this deletion synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

property display_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Display name of this resource.

done()

Method indicating whether a job has completed.

  • Returns

    True if the job has completed.

property encryption_spec(: Optional[[google.cloud.aiplatform_v1.types.encryption_spec.EncryptionSpec](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.EncryptionSpec)_ )

Customer-managed encryption key options for this Vertex AI resource.

If this is set, then all resources created by this Vertex AI resource will be encrypted with the provided encryption key.

property end_time(: Optional[datetime.datetime )

Time when the TrainingJob resource entered the PIPELINE_STATE_SUCCEEDED, PIPELINE_STATE_FAILED, PIPELINE_STATE_CANCELLED state.

property error(: Optional[google.rpc.status_pb2.Status )

Detailed error info for this TrainingJob resource. Only populated when the TrainingJob’s state is PIPELINE_STATE_FAILED or PIPELINE_STATE_CANCELLED.

property gca_resource(: [proto.message.Message](https://proto-plus-python.readthedocs.io/en/latest/reference/message.html#proto.message.Message )

The underlying resource proto representation.

classmethod get(resource_name: str, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Get Training Job for the given resource_name.

  • Parameters

    • resource_name (str) – Required. A fully-qualified resource name or ID.

    • project (str) – Optional project to retrieve training job from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional location to retrieve training job from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Custom credentials to use to upload this model. Overrides credentials set in aiplatform.init.

  • Raises

    ValueError – If the retrieved training job’s training task definition doesn’t match the custom training task definition.

  • Returns

    A Vertex AI Training Job

get_model(sync=True)

Vertex AI Model produced by this training, if one was produced.

  • Parameters

    sync (bool) – Whether to execute this method synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

  • Returns

    Vertex AI Model produced by this training

  • Return type

    model

  • Raises

    RuntimeError – If training failed or if a model was not produced by this training.

property has_failed(: [bool](https://python.readthedocs.io/en/latest/library/functions.html#bool )

Returns True if training has failed.

False otherwise.

property labels(: Dict[str, str )

User-defined labels containing metadata about this resource.

Read more about labels at https://goo.gl/xmQnxf

classmethod list(filter: Optional[str] = None, order_by: Optional[str] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

List all instances of this TrainingJob resource.

Example Usage:

aiplatform.CustomTrainingJob.list(

filter=’display_name=”experiment_a27”’,
order_by=’create_time desc’

)

  • Parameters

    • filter (str) – Optional. An expression for filtering the results of the request. For field names both snake_case and camelCase are supported.

    • order_by (str) – Optional. A comma-separated list of fields to order by, sorted in ascending order. Use “desc” after a field name for descending. Supported fields: display_name, create_time, update_time

    • project (str) – Optional. Project to retrieve list from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to retrieve list from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to retrieve list. Overrides credentials set in aiplatform.init.

  • Returns

    List[VertexAiResourceNoun] - A list of TrainingJob resource objects

property name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Name of this resource.

property resource_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Full qualified resource name.

run(dataset: google.cloud.aiplatform.datasets.image_dataset.ImageDataset, training_fraction_split: Optional[float] = None, validation_fraction_split: Optional[float] = None, test_fraction_split: Optional[float] = None, training_filter_split: Optional[str] = None, validation_filter_split: Optional[str] = None, test_filter_split: Optional[str] = None, budget_milli_node_hours: Optional[int] = None, model_display_name: Optional[str] = None, model_labels: Optional[Dict[str, str]] = None, model_id: Optional[str] = None, parent_model: Optional[str] = None, is_default_version: Optional[bool] = True, model_version_aliases: Optional[Sequence[str]] = None, model_version_description: Optional[str] = None, disable_early_stopping: bool = False, sync: bool = True, create_request_timeout: Optional[float] = None)

Runs the AutoML Image training job and returns a model.

If training on a Vertex AI dataset, you can use one of the following split configurations:

Data fraction splits: Any of training_fraction_split, validation_fraction_split and test_fraction_split may optionally be provided, they must sum to up to 1. If the provided ones sum to less than 1, the remainder is assigned to sets as decided by Vertex AI. If none of the fractions are set, by default roughly 80% of data will be used for training, 10% for validation, and 10% for test.

Data filter splits: Assigns input data to training, validation, and test sets based on the given filters, data pieces not matched by any filter are ignored. Currently only supported for Datasets containing DataItems. If any of the filters in this message are to match nothing, then they can be set as ‘-’ (the minus sign). If using filter splits, all of training_filter_split, validation_filter_split and test_filter_split must be provided. Supported only for unstructured Datasets.

  • Parameters

    • dataset (datasets.ImageDataset) – Required. The dataset within the same Project from which data will be used to train the Model. The Dataset must use schema compatible with Model being trained, and what is compatible should be described in the used TrainingPipeline’s [training_task_definition] [google.cloud.aiplatform.v1beta1.TrainingPipeline.training_task_definition]. For tabular Datasets, all their data is exported to training, to pick and choose from.

    • training_fraction_split (float) – Optional. The fraction of the input data that is to be used to train the Model. This is ignored if Dataset is not provided.

    • validation_fraction_split (float) – Optional. The fraction of the input data that is to be used to validate the Model. This is ignored if Dataset is not provided.

    • test_fraction_split (float) – Optional. The fraction of the input data that is to be used to evaluate the Model. This is ignored if Dataset is not provided.

    • training_filter_split (str) – Optional. A filter on DataItems of the Dataset. DataItems that match this filter are used to train the Model. A filter with same syntax as the one used in DatasetService.ListDataItems may be used. If a single DataItem is matched by more than one of the FilterSplit filters, then it is assigned to the first set that applies to it in the training, validation, test order. This is ignored if Dataset is not provided.

    • validation_filter_split (str) – Optional. A filter on DataItems of the Dataset. DataItems that match this filter are used to validate the Model. A filter with same syntax as the one used in DatasetService.ListDataItems may be used. If a single DataItem is matched by more than one of the FilterSplit filters, then it is assigned to the first set that applies to it in the training, validation, test order. This is ignored if Dataset is not provided.

    • test_filter_split (str) – Optional. A filter on DataItems of the Dataset. DataItems that match this filter are used to test the Model. A filter with same syntax as the one used in DatasetService.ListDataItems may be used. If a single DataItem is matched by more than one of the FilterSplit filters, then it is assigned to the first set that applies to it in the training, validation, test order. This is ignored if Dataset is not provided.

    • budget_milli_node_hours (int) – Optional. The train budget of creating this Model, expressed in milli node hours i.e. 1,000 value in this field means 1 node hour. Defaults by prediction_type: classification - For Cloud models the budget must be: 8,000 - 800,000 milli node hours (inclusive). The default value is 192,000 which represents one day in wall time, assuming 8 nodes are used. object_detection - For Cloud models the budget must be: 20,000 - 900,000 milli node hours (inclusive). The default value is 216,000 which represents one day in wall time, assuming 9 nodes are used. The training cost of the model will not exceed this budget. The final cost will be attempted to be close to the budget, though may end up being (even) noticeably smaller - at the backend’s discretion. This especially may happen when further model training ceases to provide any improvements. If the budget is set to a value known to be insufficient to train a Model for the given training set, the training won’t be attempted and will error.

    • model_display_name (str) – Optional. The display name of the managed Vertex AI Model. The name can be up to 128 characters long and can be consist of any UTF-8 characters. If not provided upon creation, the job’s display_name is used.

    • model_labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The labels with user-defined metadata to organize your Models. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information and examples of labels.

    • model_id (str) – Optional. The ID to use for the Model produced by this job, which will become the final component of the model resource name. This value may be up to 63 characters, and valid characters are [a-z0-9_-]. The first character cannot be a number or hyphen.

    • parent_model (str) – Optional. The resource name or model ID of an existing model. The new model uploaded by this job will be a version of parent_model. Only set this field when training a new version of an existing model.

    • is_default_version (bool) – Optional. When set to True, the newly uploaded model version will automatically have alias “default” included. Subsequent uses of the model produced by this job without a version specified will use this “default” version. When set to False, the “default” alias will not be moved. Actions targeting the model version produced by this job will need to specifically reference this version by ID or alias. New model uploads, i.e. version 1, will always be “default” aliased.

    • model_version_aliases (Sequence[str]) – Optional. User provided version aliases so that the model version uploaded by this job can be referenced via alias instead of auto-generated version ID. A default version alias will be created for the first version of the model. The format is [a-z][a-zA-Z0-9-]{0,126}[a-z0-9]

    • model_version_description (str) – Optional. The description of the model version being uploaded by this job.

    • disable_early_stopping – bool = False Required. If true, the entire budget is used. This disables the early stopping feature. By default, the early stopping feature is enabled, which means that training might stop before the entire training budget has been used, if further training does no longer brings significant improvement to the model.

    • sync – bool = True Whether to execute this method synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

    • create_request_timeout (float) – Optional. The timeout for the create request in seconds.

  • Returns

    The trained Vertex AI Model resource or None if training did not

      produce a Vertex AI Model.

  • Return type

    model

  • Raises

    RuntimeError – If Training job has already been run or is waiting to run.

property start_time(: Optional[datetime.datetime )

Time when the TrainingJob entered the PIPELINE_STATE_RUNNING for the first time.

property state(: Optional[[google.cloud.aiplatform_v1.types.pipeline_state.PipelineState](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.PipelineState)_ )

Current training state.

to_dict()

Returns the resource proto as a dictionary.

property update_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was last updated.

wait()

Helper method that blocks until all futures are complete.

wait_for_resource_creation()

Waits until resource has been created.

class google.cloud.aiplatform.AutoMLTabularTrainingJob(display_name: str, optimization_prediction_type: str, optimization_objective: Optional[str] = None, column_specs: Optional[Dict[str, str]] = None, column_transformations: Optional[List[Dict[str, Dict[str, str]]]] = None, optimization_objective_recall_value: Optional[float] = None, optimization_objective_precision_value: Optional[float] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None, labels: Optional[Dict[str, str]] = None, training_encryption_spec_key_name: Optional[str] = None, model_encryption_spec_key_name: Optional[str] = None)

Bases: google.cloud.aiplatform.training_jobs._TrainingJob

Constructs a AutoML Tabular Training Job.

Example usage:

job = training_jobs.AutoMLTabularTrainingJob(

display_name=”my_display_name”,
optimization_prediction_type=”classification”,
optimization_objective=”minimize-log-loss”,
column_specs={“column_1”: “auto”, “column_2”: “numeric”},
labels={‘key’: ‘value’},

)

  • Parameters

    • display_name (str) – Required. The user-defined name of this TrainingPipeline.

    • optimization_prediction_type (str) – The type of prediction the Model is to produce. “classification” - Predict one out of multiple target values is picked for each row. “regression” - Predict a value based on its relation to other values. This type is available only to columns that contain semantically numeric values, i.e. integers or floating point number, even if stored as e.g. strings.

    • optimization_objective (str) – Optional. Objective function the Model is to be optimized towards. The training task creates a Model that maximizes/minimizes the value of the objective function over the validation set.

      The supported optimization objectives depend on the prediction type, and in the case of classification also the number of distinct values in the target column (two distint values -> binary, 3 or more distinct values -> multi class). If the field is not set, the default objective function is used.

      Classification (binary): “maximize-au-roc” (default) - Maximize the area under the receiver

      operating characteristic (ROC) curve.

      ”minimize-log-loss” - Minimize log loss. “maximize-au-prc” - Maximize the area under the precision-recall curve. “maximize-precision-at-recall” - Maximize precision for a specified

      recall value.

      ”maximize-recall-at-precision” - Maximize recall for a specified

      precision value.

      Classification (multi class): “minimize-log-loss” (default) - Minimize log loss.

      Regression: “minimize-rmse” (default) - Minimize root-mean-squared error (RMSE). “minimize-mae” - Minimize mean-absolute error (MAE). “minimize-rmsle” - Minimize root-mean-squared log error (RMSLE).

    • column_specs (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. Alternative to column_transformations where the keys of the dict are column names and their respective values are one of AutoMLTabularTrainingJob.column_data_types. When creating transformation for BigQuery Struct column, the column should be flattened using “.” as the delimiter. Only columns with no child should have a transformation. If an input column has no transformations on it, such a column is ignored by the training, except for the targetColumn, which should have no transformations defined on. Only one of column_transformations or column_specs should be passed. If none of column_transformations or column_specs is passed, the local credentials being used will try setting column_specs to “auto”. To do this, the local credentials require read access to the GCS or BigQuery training data source.

    • column_transformations (List[Dict[str, **Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]]]*) – Optional. Transformations to apply to the input columns (i.e. columns other than the targetColumn). Each transformation may produce multiple result values from the column’s value, and all are used for training. When creating transformation for BigQuery Struct column, the column should be flattened using “.” as the delimiter. Only columns with no child should have a transformation. If an input column has no transformations on it, such a column is ignored by the training, except for the targetColumn, which should have no transformations defined on. Only one of column_transformations or column_specs should be passed. Consider using column_specs as column_transformations will be deprecated eventually. If none of column_transformations or column_specs is passed, the local credentials being used will try setting column_transformations to “auto”. To do this, the local credentials require read access to the GCS or BigQuery training data source.

    • optimization_objective_recall_value (float) – Optional. Required when maximize-precision-at-recall optimizationObjective was picked, represents the recall value at which the optimization is done.

      The minimum value is 0 and the maximum is 1.0.

    • optimization_objective_precision_value (float) – Optional. Required when maximize-recall-at-precision optimizationObjective was picked, represents the precision value at which the optimization is done.

      The minimum value is 0 and the maximum is 1.0.

    • project (str) – Optional. Project to run training in. Overrides project set in aiplatform.init.

    • location (str) – Optional. Location to run training in. Overrides location set in aiplatform.init.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to run call training service. Overrides credentials set in aiplatform.init.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The labels with user-defined metadata to organize TrainingPipelines. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information and examples of labels.

    • training_encryption_spec_key_name (Optional[str]) – Optional. The Cloud KMS resource identifier of the customer managed encryption key used to protect the training pipeline. Has the form: projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key. The key needs to be in the same region as where the compute resource is created.

      If set, this TrainingPipeline will be secured by this key.

      Note: Model trained by this TrainingPipeline is also secured by this key if model_to_upload is not set separately.

      Overrides encryption_spec_key_name set in aiplatform.init.

    • model_encryption_spec_key_name (Optional[str]) – Optional. The Cloud KMS resource identifier of the customer managed encryption key used to protect the model. Has the form: projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key. The key needs to be in the same region as where the compute resource is created.

      If set, the trained Model will be secured by this key.

      Overrides encryption_spec_key_name set in aiplatform.init.

  • Raises

    ValueError – If both column_transformations and column_specs were provided.

cancel()

Starts asynchronous cancellation on the TrainingJob. The server makes a best effort to cancel the job, but success is not guaranteed. On successful cancellation, the TrainingJob is not deleted; instead it becomes a job with state set to CANCELLED.

  • Raises

    RuntimeError – If this TrainingJob has not started running.

client_class()

alias of google.cloud.aiplatform.utils.PipelineClientWithOverride

property create_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was created.

delete(sync: bool = True)

Deletes this Vertex AI resource. WARNING: This deletion is permanent.

  • Parameters

    sync (bool) – Whether to execute this deletion synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

property display_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Display name of this resource.

done()

Method indicating whether a job has completed.

  • Returns

    True if the job has completed.

property encryption_spec(: Optional[[google.cloud.aiplatform_v1.types.encryption_spec.EncryptionSpec](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.EncryptionSpec)_ )

Customer-managed encryption key options for this Vertex AI resource.

If this is set, then all resources created by this Vertex AI resource will be encrypted with the provided encryption key.

property end_time(: Optional[datetime.datetime )

Time when the TrainingJob resource entered the PIPELINE_STATE_SUCCEEDED, PIPELINE_STATE_FAILED, PIPELINE_STATE_CANCELLED state.

property error(: Optional[google.rpc.status_pb2.Status )

Detailed error info for this TrainingJob resource. Only populated when the TrainingJob’s state is PIPELINE_STATE_FAILED or PIPELINE_STATE_CANCELLED.

property gca_resource(: [proto.message.Message](https://proto-plus-python.readthedocs.io/en/latest/reference/message.html#proto.message.Message )

The underlying resource proto representation.

classmethod get(resource_name: str, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Get Training Job for the given resource_name.

  • Parameters

    • resource_name (str) – Required. A fully-qualified resource name or ID.

    • project (str) – Optional project to retrieve training job from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional location to retrieve training job from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Custom credentials to use to upload this model. Overrides credentials set in aiplatform.init.

  • Raises

    ValueError – If the retrieved training job’s training task definition doesn’t match the custom training task definition.

  • Returns

    A Vertex AI Training Job

static get_auto_column_specs(dataset: google.cloud.aiplatform.datasets.tabular_dataset.TabularDataset, target_column: str)

Returns a dict with all non-target columns as keys and ‘auto’ as values.

Example usage:

column_specs = training_jobs.AutoMLTabularTrainingJob.get_auto_column_specs(

dataset=my_dataset,
target_column=”my_target_column”,

)

  • Parameters

    • dataset (datasets.TabularDataset) – Required. Intended dataset.

    • target_column (str) – Required. Intended target column.

  • Returns

    Dict[str, str]

      Column names as keys and ‘auto’ as values

get_model(sync=True)

Vertex AI Model produced by this training, if one was produced.

  • Parameters

    sync (bool) – Whether to execute this method synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

  • Returns

    Vertex AI Model produced by this training

  • Return type

    model

  • Raises

    RuntimeError – If training failed or if a model was not produced by this training.

property has_failed(: [bool](https://python.readthedocs.io/en/latest/library/functions.html#bool )

Returns True if training has failed.

False otherwise.

property labels(: Dict[str, str )

User-defined labels containing metadata about this resource.

Read more about labels at https://goo.gl/xmQnxf

classmethod list(filter: Optional[str] = None, order_by: Optional[str] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

List all instances of this TrainingJob resource.

Example Usage:

aiplatform.CustomTrainingJob.list(

filter=’display_name=”experiment_a27”’,
order_by=’create_time desc’

)

  • Parameters

    • filter (str) – Optional. An expression for filtering the results of the request. For field names both snake_case and camelCase are supported.

    • order_by (str) – Optional. A comma-separated list of fields to order by, sorted in ascending order. Use “desc” after a field name for descending. Supported fields: display_name, create_time, update_time

    • project (str) – Optional. Project to retrieve list from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to retrieve list from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to retrieve list. Overrides credentials set in aiplatform.init.

  • Returns

    List[VertexAiResourceNoun] - A list of TrainingJob resource objects

property name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Name of this resource.

property resource_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Full qualified resource name.

run(dataset: google.cloud.aiplatform.datasets.tabular_dataset.TabularDataset, target_column: str, training_fraction_split: Optional[float] = None, validation_fraction_split: Optional[float] = None, test_fraction_split: Optional[float] = None, predefined_split_column_name: Optional[str] = None, timestamp_split_column_name: Optional[str] = None, weight_column: Optional[str] = None, budget_milli_node_hours: int = 1000, model_display_name: Optional[str] = None, model_labels: Optional[Dict[str, str]] = None, model_id: Optional[str] = None, parent_model: Optional[str] = None, is_default_version: Optional[bool] = True, model_version_aliases: Optional[Sequence[str]] = None, model_version_description: Optional[str] = None, disable_early_stopping: bool = False, export_evaluated_data_items: bool = False, export_evaluated_data_items_bigquery_destination_uri: Optional[str] = None, export_evaluated_data_items_override_destination: bool = False, additional_experiments: Optional[List[str]] = None, sync: bool = True, create_request_timeout: Optional[float] = None)

Runs the training job and returns a model.

If training on a Vertex AI dataset, you can use one of the following split configurations:

Data fraction splits:
Any of `training_fraction_split`, `validation_fraction_split` and
`test_fraction_split` may optionally be provided, they must sum to up to 1. If
the provided ones sum to less than 1, the remainder is assigned to sets as
decided by Vertex AI. If none of the fractions are set, by default roughly 80%
of data will be used for training, 10% for validation, and 10% for test.

Predefined splits:
Assigns input data to training, validation, and test sets based on the value of a provided key.
If using predefined splits, `predefined_split_column_name` must be provided.
Supported only for tabular Datasets.

Timestamp splits:
Assigns input data to training, validation, and test sets
based on a provided timestamps. The youngest data pieces are
assigned to training set, next to validation set, and the oldest
to the test set.
Supported only for tabular Datasets.

  • Parameters

    • dataset (datasets.TabularDataset) – Required. The dataset within the same Project from which data will be used to train the Model. The Dataset must use schema compatible with Model being trained, and what is compatible should be described in the used TrainingPipeline’s [training_task_definition] [google.cloud.aiplatform.v1beta1.TrainingPipeline.training_task_definition]. For tabular Datasets, all their data is exported to training, to pick and choose from.

    • target_column (str) – Required. The name of the column values of which the Model is to predict.

    • training_fraction_split (float) – Optional. The fraction of the input data that is to be used to train the Model. This is ignored if Dataset is not provided.

    • validation_fraction_split (float) – Optional. The fraction of the input data that is to be used to validate the Model. This is ignored if Dataset is not provided.

    • test_fraction_split (float) – Optional. The fraction of the input data that is to be used to evaluate the Model. This is ignored if Dataset is not provided.

    • predefined_split_column_name (str) – Optional. The key is a name of one of the Dataset’s data columns. The value of the key (either the label’s value or value in the column) must be one of {training, validation, test}, and it defines to which set the given piece of data is assigned. If for a piece of data the key is not present or has an invalid value, that piece is ignored by the pipeline.

      Supported only for tabular and time series Datasets.

    • timestamp_split_column_name (str) – Optional. The key is a name of one of the Dataset’s data columns. The value of the key values of the key (the values in the column) must be in RFC 3339 date-time format, where time-offset = “Z” (e.g. 1985-04-12T23:20:50.52Z). If for a piece of data the key is not present or has an invalid value, that piece is ignored by the pipeline. Supported only for tabular and time series Datasets. This parameter must be used with training_fraction_split, validation_fraction_split, and test_fraction_split.

    • weight_column (str) – Optional. Name of the column that should be used as the weight column. Higher values in this column give more importance to the row during Model training. The column must have numeric values between 0 and 10000 inclusively, and 0 value means that the row is ignored. If the weight column field is not set, then all rows are assumed to have equal weight of 1.

    • budget_milli_node_hours (int) – Optional. The train budget of creating this Model, expressed in milli node hours i.e. 1,000 value in this field means 1 node hour. The training cost of the model will not exceed this budget. The final cost will be attempted to be close to the budget, though may end up being (even) noticeably smaller - at the backend’s discretion. This especially may happen when further model training ceases to provide any improvements. If the budget is set to a value known to be insufficient to train a Model for the given training set, the training won’t be attempted and will error. The minimum value is 1000 and the maximum is 72000.

    • model_display_name (str) – Optional. If the script produces a managed Vertex AI Model. The display name of the Model. The name can be up to 128 characters long and can be consist of any UTF-8 characters.

      If not provided upon creation, the job’s display_name is used.

    • model_labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The labels with user-defined metadata to organize your Models. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information and examples of labels.

    • model_id (str) – Optional. The ID to use for the Model produced by this job, which will become the final component of the model resource name. This value may be up to 63 characters, and valid characters are [a-z0-9_-]. The first character cannot be a number or hyphen.

    • parent_model (str) – Optional. The resource name or model ID of an existing model. The new model uploaded by this job will be a version of parent_model.

      Only set this field when training a new version of an existing model.

    • is_default_version (bool) – Optional. When set to True, the newly uploaded model version will automatically have alias “default” included. Subsequent uses of the model produced by this job without a version specified will use this “default” version.

      When set to False, the “default” alias will not be moved. Actions targeting the model version produced by this job will need to specifically reference this version by ID or alias.

      New model uploads, i.e. version 1, will always be “default” aliased.

    • model_version_aliases (Sequence[str]) – Optional. User provided version aliases so that the model version uploaded by this job can be referenced via alias instead of auto-generated version ID. A default version alias will be created for the first version of the model.

      The format is [a-z][a-zA-Z0-9-]{0,126}[a-z0-9]

    • model_version_description (str) – Optional. The description of the model version being uploaded by this job.

    • disable_early_stopping (bool) – Required. If true, the entire budget is used. This disables the early stopping feature. By default, the early stopping feature is enabled, which means that training might stop before the entire training budget has been used, if further training does no longer brings significant improvement to the model.

    • export_evaluated_data_items (bool) – Whether to export the test set predictions to a BigQuery table. If False, then the export is not performed.

    • export_evaluated_data_items_bigquery_destination_uri (string) – Optional. URI of desired destination BigQuery table for exported test set predictions.

      Expected format: bq://<project_id>:<dataset_id>:<table>

      If not specified, then results are exported to the following auto-created BigQuery table: <project_id>:export_evaluated_examples_<model_name>_<yyyy_MM_dd'T'HH_mm_ss_SSS'Z'>.evaluated_examples

      Applies only if [export_evaluated_data_items] is True.

    • export_evaluated_data_items_override_destination (bool) – Whether to override the contents of [export_evaluated_data_items_bigquery_destination_uri], if the table exists, for exported test set predictions. If False, and the table exists, then the training job will fail.

      Applies only if [export_evaluated_data_items] is True and [export_evaluated_data_items_bigquery_destination_uri] is specified.

    • additional_experiments (List[str]) – Optional. Additional experiment flags for the automl tables training.

    • sync (bool) – Whether to execute this method synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

    • create_request_timeout (float) – Optional. The timeout for the create request in seconds.

  • Returns

    The trained Vertex AI Model resource or None if training did not

      produce a Vertex AI Model.

  • Return type

    model

  • Raises

    RuntimeError – If Training job has already been run or is waiting to run.

property start_time(: Optional[datetime.datetime )

Time when the TrainingJob entered the PIPELINE_STATE_RUNNING for the first time.

property state(: Optional[[google.cloud.aiplatform_v1.types.pipeline_state.PipelineState](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.PipelineState)_ )

Current training state.

to_dict()

Returns the resource proto as a dictionary.

property update_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was last updated.

wait()

Helper method that blocks until all futures are complete.

wait_for_resource_creation()

Waits until resource has been created.

class google.cloud.aiplatform.AutoMLTextTrainingJob(display_name: str, prediction_type: str, multi_label: bool = False, sentiment_max: int = 10, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None, labels: Optional[Dict[str, str]] = None, training_encryption_spec_key_name: Optional[str] = None, model_encryption_spec_key_name: Optional[str] = None)

Bases: google.cloud.aiplatform.training_jobs._TrainingJob

Constructs a AutoML Text Training Job.

  • Parameters

    • display_name (str) – Required. The user-defined name of this TrainingPipeline.

    • prediction_type (str) – The type of prediction the Model is to produce, one of:

      ”classification” - A classification model analyzes text data and

        returns a list of categories that apply to the text found in the data.
  Vertex AI offers both single-label and multi-label text classification models.

      ”extraction” - An entity extraction model inspects text data

        for known entities referenced in the data and
  labels those entities in the text.

      ”sentiment” - A sentiment analysis model inspects text data and identifies the

        prevailing emotional opinion within it, especially to determine a writer’s attitude
  as positive, negative, or neutral.

    • multi_label (bool) – Required and only applicable for text classification task. If false, a single-label (multi-class) Model will be trained (i.e. assuming that for each text snippet just up to one annotation may be applicable). If true, a multi-label Model will be trained (i.e. assuming that for each text snippet multiple annotations may be applicable).

    • sentiment_max (int) – Required and only applicable for sentiment task. A sentiment is expressed as an integer ordinal, where higher value means a more positive sentiment. The range of sentiments that will be used is between 0 and sentimentMax (inclusive on both ends), and all the values in the range must be represented in the dataset before a model can be created. Only the Annotations with this sentimentMax will be used for training. sentimentMax value must be between 1 and 10 (inclusive).

    • project (str) – Optional. Project to run training in. Overrides project set in aiplatform.init.

    • location (str) – Optional. Location to run training in. Overrides location set in aiplatform.init.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to run call training service. Overrides credentials set in aiplatform.init.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The labels with user-defined metadata to organize TrainingPipelines. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information and examples of labels.

    • training_encryption_spec_key_name (Optional[str]) – Optional. The Cloud KMS resource identifier of the customer managed encryption key used to protect the training pipeline. Has the form: projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key. The key needs to be in the same region as where the compute resource is created.

      If set, this TrainingPipeline will be secured by this key.

      Note: Model trained by this TrainingPipeline is also secured by this key if model_to_upload is not set separately.

      Overrides encryption_spec_key_name set in aiplatform.init.

    • model_encryption_spec_key_name (Optional[str]) – Optional. The Cloud KMS resource identifier of the customer managed encryption key used to protect the model. Has the form: projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key. The key needs to be in the same region as where the compute resource is created.

      If set, the trained Model will be secured by this key.

      Overrides encryption_spec_key_name set in aiplatform.init.

cancel()

Starts asynchronous cancellation on the TrainingJob. The server makes a best effort to cancel the job, but success is not guaranteed. On successful cancellation, the TrainingJob is not deleted; instead it becomes a job with state set to CANCELLED.

  • Raises

    RuntimeError – If this TrainingJob has not started running.

client_class()

alias of google.cloud.aiplatform.utils.PipelineClientWithOverride

property create_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was created.

delete(sync: bool = True)

Deletes this Vertex AI resource. WARNING: This deletion is permanent.

  • Parameters

    sync (bool) – Whether to execute this deletion synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

property display_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Display name of this resource.

done()

Method indicating whether a job has completed.

  • Returns

    True if the job has completed.

property encryption_spec(: Optional[[google.cloud.aiplatform_v1.types.encryption_spec.EncryptionSpec](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.EncryptionSpec)_ )

Customer-managed encryption key options for this Vertex AI resource.

If this is set, then all resources created by this Vertex AI resource will be encrypted with the provided encryption key.

property end_time(: Optional[datetime.datetime )

Time when the TrainingJob resource entered the PIPELINE_STATE_SUCCEEDED, PIPELINE_STATE_FAILED, PIPELINE_STATE_CANCELLED state.

property error(: Optional[google.rpc.status_pb2.Status )

Detailed error info for this TrainingJob resource. Only populated when the TrainingJob’s state is PIPELINE_STATE_FAILED or PIPELINE_STATE_CANCELLED.

property gca_resource(: [proto.message.Message](https://proto-plus-python.readthedocs.io/en/latest/reference/message.html#proto.message.Message )

The underlying resource proto representation.

classmethod get(resource_name: str, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Get Training Job for the given resource_name.

  • Parameters

    • resource_name (str) – Required. A fully-qualified resource name or ID.

    • project (str) – Optional project to retrieve training job from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional location to retrieve training job from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Custom credentials to use to upload this model. Overrides credentials set in aiplatform.init.

  • Raises

    ValueError – If the retrieved training job’s training task definition doesn’t match the custom training task definition.

  • Returns

    A Vertex AI Training Job

get_model(sync=True)

Vertex AI Model produced by this training, if one was produced.

  • Parameters

    sync (bool) – Whether to execute this method synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

  • Returns

    Vertex AI Model produced by this training

  • Return type

    model

  • Raises

    RuntimeError – If training failed or if a model was not produced by this training.

property has_failed(: [bool](https://python.readthedocs.io/en/latest/library/functions.html#bool )

Returns True if training has failed.

False otherwise.

property labels(: Dict[str, str )

User-defined labels containing metadata about this resource.

Read more about labels at https://goo.gl/xmQnxf

classmethod list(filter: Optional[str] = None, order_by: Optional[str] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

List all instances of this TrainingJob resource.

Example Usage:

aiplatform.CustomTrainingJob.list(

filter=’display_name=”experiment_a27”’,
order_by=’create_time desc’

)

  • Parameters

    • filter (str) – Optional. An expression for filtering the results of the request. For field names both snake_case and camelCase are supported.

    • order_by (str) – Optional. A comma-separated list of fields to order by, sorted in ascending order. Use “desc” after a field name for descending. Supported fields: display_name, create_time, update_time

    • project (str) – Optional. Project to retrieve list from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to retrieve list from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to retrieve list. Overrides credentials set in aiplatform.init.

  • Returns

    List[VertexAiResourceNoun] - A list of TrainingJob resource objects

property name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Name of this resource.

property resource_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Full qualified resource name.

run(dataset: google.cloud.aiplatform.datasets.text_dataset.TextDataset, training_fraction_split: Optional[float] = None, validation_fraction_split: Optional[float] = None, test_fraction_split: Optional[float] = None, training_filter_split: Optional[str] = None, validation_filter_split: Optional[str] = None, test_filter_split: Optional[str] = None, model_display_name: Optional[str] = None, model_labels: Optional[Dict[str, str]] = None, model_id: Optional[str] = None, parent_model: Optional[str] = None, is_default_version: Optional[bool] = True, model_version_aliases: Optional[Sequence[str]] = None, model_version_description: Optional[str] = None, sync: bool = True, create_request_timeout: Optional[float] = None)

Runs the training job and returns a model.

If training on a Vertex AI dataset, you can use one of the following split configurations:

Data fraction splits:
Any of `training_fraction_split`, `validation_fraction_split` and
`test_fraction_split` may optionally be provided, they must sum to up to 1. If
the provided ones sum to less than 1, the remainder is assigned to sets as
decided by Vertex AI. If none of the fractions are set, by default roughly 80%
of data will be used for training, 10% for validation, and 10% for test.

Data filter splits:
Assigns input data to training, validation, and test sets
based on the given filters, data pieces not matched by any
filter are ignored. Currently only supported for Datasets
containing DataItems.
If any of the filters in this message are to match nothing, then
they can be set as ‘-’ (the minus sign).
If using filter splits, all of `training_filter_split`, `validation_filter_split` and
`test_filter_split` must be provided.
Supported only for unstructured Datasets.

  • Parameters

    • dataset (datasets.TextDataset) – Required. The dataset within the same Project from which data will be used to train the Model. The Dataset must use schema compatible with Model being trained, and what is compatible should be described in the used TrainingPipeline’s [training_task_definition] [google.cloud.aiplatform.v1beta1.TrainingPipeline.training_task_definition].

    • training_fraction_split (float) – Optional. The fraction of the input data that is to be used to train the Model. This is ignored if Dataset is not provided.

    • validation_fraction_split (float) – Optional. The fraction of the input data that is to be used to validate the Model. This is ignored if Dataset is not provided.

    • test_fraction_split (float) – Optional. The fraction of the input data that is to be used to evaluate the Model. This is ignored if Dataset is not provided.

    • training_filter_split (str) – Optional. A filter on DataItems of the Dataset. DataItems that match this filter are used to train the Model. A filter with same syntax as the one used in DatasetService.ListDataItems may be used. If a single DataItem is matched by more than one of the FilterSplit filters, then it is assigned to the first set that applies to it in the training, validation, test order. This is ignored if Dataset is not provided.

    • validation_filter_split (str) – Optional. A filter on DataItems of the Dataset. DataItems that match this filter are used to validate the Model. A filter with same syntax as the one used in DatasetService.ListDataItems may be used. If a single DataItem is matched by more than one of the FilterSplit filters, then it is assigned to the first set that applies to it in the training, validation, test order. This is ignored if Dataset is not provided.

    • test_filter_split (str) – Optional. A filter on DataItems of the Dataset. DataItems that match this filter are used to test the Model. A filter with same syntax as the one used in DatasetService.ListDataItems may be used. If a single DataItem is matched by more than one of the FilterSplit filters, then it is assigned to the first set that applies to it in the training, validation, test order. This is ignored if Dataset is not provided.

    • model_display_name (str) – Optional. The display name of the managed Vertex AI Model. The name can be up to 128 characters long and can consist of any UTF-8 characters.

      If not provided upon creation, the job’s display_name is used.

    • model_labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The labels with user-defined metadata to organize your Models. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information and examples of labels..

    • model_id (str) – Optional. The ID to use for the Model produced by this job, which will become the final component of the model resource name. This value may be up to 63 characters, and valid characters are [a-z0-9_-]. The first character cannot be a number or hyphen.

    • parent_model (str) – Optional. The resource name or model ID of an existing model. The new model uploaded by this job will be a version of parent_model.

      Only set this field when training a new version of an existing model.

    • is_default_version (bool) – Optional. When set to True, the newly uploaded model version will automatically have alias “default” included. Subsequent uses of the model produced by this job without a version specified will use this “default” version.

      When set to False, the “default” alias will not be moved. Actions targeting the model version produced by this job will need to specifically reference this version by ID or alias.

      New model uploads, i.e. version 1, will always be “default” aliased.

    • model_version_aliases (Sequence[str]) – Optional. User provided version aliases so that the model version uploaded by this job can be referenced via alias instead of auto-generated version ID. A default version alias will be created for the first version of the model.

      The format is [a-z][a-zA-Z0-9-]{0,126}[a-z0-9]

    • model_version_description (str) – Optional. The description of the model version being uploaded by this job.

    • sync (bool) – Whether to execute this method synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

    • create_request_timeout (float) – Optional. The timeout for the create request in seconds

  • Returns

    The trained Vertex AI Model resource.

  • Return type

    model

  • Raises

    RuntimeError – If Training job has already been run or is waiting to run.

property start_time(: Optional[datetime.datetime )

Time when the TrainingJob entered the PIPELINE_STATE_RUNNING for the first time.

property state(: Optional[[google.cloud.aiplatform_v1.types.pipeline_state.PipelineState](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.PipelineState)_ )

Current training state.

to_dict()

Returns the resource proto as a dictionary.

property update_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was last updated.

wait()

Helper method that blocks until all futures are complete.

wait_for_resource_creation()

Waits until resource has been created.

class google.cloud.aiplatform.AutoMLVideoTrainingJob(display_name: Optional[str] = None, prediction_type: str = 'classification', model_type: str = 'CLOUD', project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None, labels: Optional[Dict[str, str]] = None, training_encryption_spec_key_name: Optional[str] = None, model_encryption_spec_key_name: Optional[str] = None)

Bases: google.cloud.aiplatform.training_jobs._TrainingJob

Constructs a AutoML Video Training Job.

  • Parameters

    • display_name (str) – Required. The user-defined name of this TrainingPipeline.

    • prediction_type (str) – The type of prediction the Model is to produce, one of:

      ”classification” - A video classification model classifies shots

        and segments in your videos according to your own defined labels.

      ”object_tracking” - A video object tracking model detects and tracks

        multiple objects in shots and segments. You can use these
  models to track objects in your videos according to your
  own pre-defined, custom labels.

      ”action_recognition” - A video action recognition model pinpoints

        the location of actions with short temporal durations (~1 second).

    • model_type – str = “CLOUD” Required. One of the following:

      ”CLOUD” - available for “classification”, “object_tracking” and “action_recognition”

      A Model best tailored to be used within Google Cloud,
and which cannot be exported.

      ”MOBILE_VERSATILE_1” - available for “classification”, “object_tracking” and “action_recognition”

      A model that, in addition to being available within Google
Cloud, can also be exported (see ModelService.ExportModel)
as a TensorFlow or TensorFlow Lite model and used on a
mobile or edge device with afterwards.

      ”MOBILE_CORAL_VERSATILE_1” - available only for “object_tracking”

      A versatile model that is meant to be exported (see
ModelService.ExportModel) and used on a Google Coral device.

      ”MOBILE_CORAL_LOW_LATENCY_1” - available only for “object_tracking”

      A model that trades off quality for low latency, to be
exported (see ModelService.ExportModel) and used on a
Google Coral device.

      ”MOBILE_JETSON_VERSATILE_1” - available only for “object_tracking”

      A versatile model that is meant to be exported (see
ModelService.ExportModel) and used on an NVIDIA Jetson device.

      ”MOBILE_JETSON_LOW_LATENCY_1” - available only for “object_tracking”

      A model that trades off quality for low latency, to be
exported (see ModelService.ExportModel) and used on an
NVIDIA Jetson device.

    • project (str) – Optional. Project to run training in. Overrides project set in aiplatform.init.

    • location (str) – Optional. Location to run training in. Overrides location set in aiplatform.init.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to run call training service. Overrides credentials set in aiplatform.init.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The labels with user-defined metadata to organize TrainingPipelines. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information and examples of labels.

    • training_encryption_spec_key_name (Optional[str]) – Optional. The Cloud KMS resource identifier of the customer managed encryption key used to protect the training pipeline. Has the form: projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key. The key needs to be in the same region as where the compute resource is created.

      If set, this TrainingPipeline will be secured by this key.

      Note: Model trained by this TrainingPipeline is also secured by this key if model_to_upload is not set separately.

      Overrides encryption_spec_key_name set in aiplatform.init.

    • model_encryption_spec_key_name (Optional[str]) – Optional. The Cloud KMS resource identifier of the customer managed encryption key used to protect the model. Has the form: projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key. The key needs to be in the same region as where the compute resource is created.

      If set, the trained Model will be secured by this key.

      Overrides encryption_spec_key_name set in aiplatform.init.

  • Raises

    ValueError – When an invalid prediction_type and/or model_type is provided.

cancel()

Starts asynchronous cancellation on the TrainingJob. The server makes a best effort to cancel the job, but success is not guaranteed. On successful cancellation, the TrainingJob is not deleted; instead it becomes a job with state set to CANCELLED.

  • Raises

    RuntimeError – If this TrainingJob has not started running.

client_class()

alias of google.cloud.aiplatform.utils.PipelineClientWithOverride

property create_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was created.

delete(sync: bool = True)

Deletes this Vertex AI resource. WARNING: This deletion is permanent.

  • Parameters

    sync (bool) – Whether to execute this deletion synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

property display_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Display name of this resource.

done()

Method indicating whether a job has completed.

  • Returns

    True if the job has completed.

property encryption_spec(: Optional[[google.cloud.aiplatform_v1.types.encryption_spec.EncryptionSpec](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.EncryptionSpec)_ )

Customer-managed encryption key options for this Vertex AI resource.

If this is set, then all resources created by this Vertex AI resource will be encrypted with the provided encryption key.

property end_time(: Optional[datetime.datetime )

Time when the TrainingJob resource entered the PIPELINE_STATE_SUCCEEDED, PIPELINE_STATE_FAILED, PIPELINE_STATE_CANCELLED state.

property error(: Optional[google.rpc.status_pb2.Status )

Detailed error info for this TrainingJob resource. Only populated when the TrainingJob’s state is PIPELINE_STATE_FAILED or PIPELINE_STATE_CANCELLED.

property gca_resource(: [proto.message.Message](https://proto-plus-python.readthedocs.io/en/latest/reference/message.html#proto.message.Message )

The underlying resource proto representation.

classmethod get(resource_name: str, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Get Training Job for the given resource_name.

  • Parameters

    • resource_name (str) – Required. A fully-qualified resource name or ID.

    • project (str) – Optional project to retrieve training job from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional location to retrieve training job from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Custom credentials to use to upload this model. Overrides credentials set in aiplatform.init.

  • Raises

    ValueError – If the retrieved training job’s training task definition doesn’t match the custom training task definition.

  • Returns

    A Vertex AI Training Job

get_model(sync=True)

Vertex AI Model produced by this training, if one was produced.

  • Parameters

    sync (bool) – Whether to execute this method synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

  • Returns

    Vertex AI Model produced by this training

  • Return type

    model

  • Raises

    RuntimeError – If training failed or if a model was not produced by this training.

property has_failed(: [bool](https://python.readthedocs.io/en/latest/library/functions.html#bool )

Returns True if training has failed.

False otherwise.

property labels(: Dict[str, str )

User-defined labels containing metadata about this resource.

Read more about labels at https://goo.gl/xmQnxf

classmethod list(filter: Optional[str] = None, order_by: Optional[str] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

List all instances of this TrainingJob resource.

Example Usage:

aiplatform.CustomTrainingJob.list(

filter=’display_name=”experiment_a27”’,
order_by=’create_time desc’

)

  • Parameters

    • filter (str) – Optional. An expression for filtering the results of the request. For field names both snake_case and camelCase are supported.

    • order_by (str) – Optional. A comma-separated list of fields to order by, sorted in ascending order. Use “desc” after a field name for descending. Supported fields: display_name, create_time, update_time

    • project (str) – Optional. Project to retrieve list from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to retrieve list from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to retrieve list. Overrides credentials set in aiplatform.init.

  • Returns

    List[VertexAiResourceNoun] - A list of TrainingJob resource objects

property name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Name of this resource.

property resource_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Full qualified resource name.

run(dataset: google.cloud.aiplatform.datasets.video_dataset.VideoDataset, training_fraction_split: Optional[float] = None, test_fraction_split: Optional[float] = None, training_filter_split: Optional[str] = None, test_filter_split: Optional[str] = None, model_display_name: Optional[str] = None, model_labels: Optional[Dict[str, str]] = None, model_id: Optional[str] = None, parent_model: Optional[str] = None, is_default_version: Optional[bool] = True, model_version_aliases: Optional[Sequence[str]] = None, model_version_description: Optional[str] = None, sync: bool = True, create_request_timeout: Optional[float] = None)

Runs the AutoML Video training job and returns a model.

If training on a Vertex AI dataset, you can use one of the following split configurations:

Data fraction splits:
`training_fraction_split`, and `test_fraction_split` may optionally
be provided, they must sum to up to 1. If none of the fractions are set,
by default roughly 80% of data will be used for training, and 20% for test.

Data filter splits:
Assigns input data to training, validation, and test sets
based on the given filters, data pieces not matched by any
filter are ignored. Currently only supported for Datasets
containing DataItems.
If any of the filters in this message are to match nothing, then
they can be set as ‘-’ (the minus sign).
If using filter splits, all of `training_filter_split`, `validation_filter_split` and
`test_filter_split` must be provided.
Supported only for unstructured Datasets.

  • Parameters

    • dataset (datasets.VideoDataset) – Required. The dataset within the same Project from which data will be used to train the Model. The Dataset must use schema compatible with Model being trained, and what is compatible should be described in the used TrainingPipeline’s [training_task_definition] [google.cloud.aiplatform.v1beta1.TrainingPipeline.training_task_definition]. For tabular Datasets, all their data is exported to training, to pick and choose from.

    • training_fraction_split (float) – Optional. The fraction of the input data that is to be used to train the Model. This is ignored if Dataset is not provided.

    • test_fraction_split (float) – Optional. The fraction of the input data that is to be used to evaluate the Model. This is ignored if Dataset is not provided.

    • training_filter_split (str) – Optional. A filter on DataItems of the Dataset. DataItems that match this filter are used to train the Model. A filter with same syntax as the one used in DatasetService.ListDataItems may be used. If a single DataItem is matched by more than one of the FilterSplit filters, then it is assigned to the first set that applies to it in the training, validation, test order. This is ignored if Dataset is not provided.

    • test_filter_split (str) – Optional. A filter on DataItems of the Dataset. DataItems that match this filter are used to test the Model. A filter with same syntax as the one used in DatasetService.ListDataItems may be used. If a single DataItem is matched by more than one of the FilterSplit filters, then it is assigned to the first set that applies to it in the training, validation, test order. This is ignored if Dataset is not provided.

    • model_display_name (str) – Optional. The display name of the managed Vertex AI Model. The name can be up to 128 characters long and can be consist of any UTF-8 characters. If not provided upon creation, the job’s display_name is used.

    • model_labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The labels with user-defined metadata to organize your Models. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information and examples of labels.

    • model_id (str) – Optional. The ID to use for the Model produced by this job, which will become the final component of the model resource name. This value may be up to 63 characters, and valid characters are [a-z0-9_-]. The first character cannot be a number or hyphen.

    • parent_model (str) – Optional. The resource name or model ID of an existing model. The new model uploaded by this job will be a version of parent_model.

      Only set this field when training a new version of an existing model.

    • is_default_version (bool) – Optional. When set to True, the newly uploaded model version will automatically have alias “default” included. Subsequent uses of the model produced by this job without a version specified will use this “default” version.

      When set to False, the “default” alias will not be moved. Actions targeting the model version produced by this job will need to specifically reference this version by ID or alias.

      New model uploads, i.e. version 1, will always be “default” aliased.

    • model_version_aliases (Sequence[str]) – Optional. User provided version aliases so that the model version uploaded by this job can be referenced via alias instead of auto-generated version ID. A default version alias will be created for the first version of the model.

      The format is [a-z][a-zA-Z0-9-]{0,126}[a-z0-9]

    • model_version_description (str) – Optional. The description of the model version being uploaded by this job.

    • sync – bool = True Whether to execute this method synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

    • create_request_timeout (float) – Optional. The timeout for the create request in seconds.

  • Returns

    The trained Vertex AI Model resource or None if training did not

      produce a Vertex AI Model.

  • Return type

    model

  • Raises

    RuntimeError – If Training job has already been run or is waiting to run.

property start_time(: Optional[datetime.datetime )

Time when the TrainingJob entered the PIPELINE_STATE_RUNNING for the first time.

property state(: Optional[[google.cloud.aiplatform_v1.types.pipeline_state.PipelineState](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.PipelineState)_ )

Current training state.

to_dict()

Returns the resource proto as a dictionary.

property update_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was last updated.

wait()

Helper method that blocks until all futures are complete.

wait_for_resource_creation()

Waits until resource has been created.

class google.cloud.aiplatform.BatchPredictionJob(batch_prediction_job_name: str, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Bases: google.cloud.aiplatform.jobs._Job

Retrieves a BatchPredictionJob resource and instantiates its representation.

  • Parameters

    • batch_prediction_job_name (str) – Required. A fully-qualified BatchPredictionJob resource name or ID. Example: “projects/…/locations/…/batchPredictionJobs/456” or “456” when project and location are initialized or passed.

    • project – Optional[str] = None, Optional. project to retrieve BatchPredictionJob from. If not set, project set in aiplatform.init will be used.

    • location – Optional[str] = None, Optional. location to retrieve BatchPredictionJob from. If not set, location set in aiplatform.init will be used.

    • credentials – Optional[auth_credentials.Credentials] = None, Custom credentials to use. If not set, credentials set in aiplatform.init will be used.

cancel()

Cancels this Job.

Success of cancellation is not guaranteed. Use Job.state property to verify if cancellation was successful.

client_class()

alias of google.cloud.aiplatform.utils.JobClientWithOverride

property completion_stats(: Optional[[google.cloud.aiplatform_v1.types.completion_stats.CompletionStats](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.CompletionStats)_ )

Statistics on completed and failed prediction instances.

classmethod create(job_display_name: str, model_name: Union[str, google.cloud.aiplatform.models.Model], instances_format: str = 'jsonl', predictions_format: str = 'jsonl', gcs_source: Optional[Union[str, Sequence[str]]] = None, bigquery_source: Optional[str] = None, gcs_destination_prefix: Optional[str] = None, bigquery_destination_prefix: Optional[str] = None, model_parameters: Optional[Dict] = None, machine_type: Optional[str] = None, accelerator_type: Optional[str] = None, accelerator_count: Optional[int] = None, starting_replica_count: Optional[int] = None, max_replica_count: Optional[int] = None, generate_explanation: Optional[bool] = False, explanation_metadata: Optional[google.cloud.aiplatform_v1.types.explanation_metadata.ExplanationMetadata] = None, explanation_parameters: Optional[google.cloud.aiplatform_v1.types.explanation.ExplanationParameters] = None, labels: Optional[Dict[str, str]] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None, encryption_spec_key_name: Optional[str] = None, sync: bool = True, create_request_timeout: Optional[float] = None, batch_size: Optional[int] = None, model_monitoring_objective_config: Optional[google.cloud.aiplatform.model_monitoring.objective.ObjectiveConfig] = None, model_monitoring_alert_config: Optional[google.cloud.aiplatform.model_monitoring.alert.AlertConfig] = None, analysis_instance_schema_uri: Optional[str] = None, service_account: Optional[str] = None)

Create a batch prediction job.

  • Parameters

    • job_display_name (str) – Required. The user-defined name of the BatchPredictionJob. The name can be up to 128 characters long and can be consist of any UTF-8 characters.

    • model_name (Union[str, **aiplatform.Model]) – Required. A fully-qualified model resource name or model ID. Example: “projects/123/locations/us-central1/models/456” or “456” when project and location are initialized or passed. May optionally contain a version ID or alias in {model_name}@{version} form.

      Or an instance of aiplatform.Model.

    • instances_format (str) – Required. The format in which instances are provided. Must be one of the formats listed in Model.supported_input_storage_formats. Default is “jsonl” when using gcs_source. If a bigquery_source is provided, this is overridden to “bigquery”.

    • predictions_format (str) – Required. The format in which Vertex AI outputs the predictions, must be one of the formats specified in Model.supported_output_storage_formats. Default is “jsonl” when using gcs_destination_prefix. If a bigquery_destination_prefix is provided, this is overridden to “bigquery”.

    • gcs_source (Optional[Sequence[str]]) – Google Cloud Storage URI(-s) to your instances to run batch prediction on. They must match instances_format.

    • bigquery_source (Optional[str]) – BigQuery URI to a table, up to 2000 characters long. For example: bq://projectId.bqDatasetId.bqTableId

    • gcs_destination_prefix (Optional[str]) – The Google Cloud Storage location of the directory where the output is to be written to. In the given directory a new directory is created. Its name is prediction-<model-display-name>-<job-create-time>, where timestamp is in YYYY-MM-DDThh:mm:ss.sssZ ISO-8601 format. Inside of it files predictions_0001.<extension>, predictions_0002.<extension>, …, predictions_N.<extension> are created where <extension> depends on chosen predictions_format, and N may equal 0001 and depends on the total number of successfully predicted instances. If the Model has both instance and prediction schemata defined then each such file contains predictions as per the predictions_format. If prediction for any instance failed (partially or completely), then an additional errors_0001.<extension>, errors_0002.<extension>,…, errors_N.<extension> files are created (N depends on total number of failed predictions). These files contain the failed instances, as per their schema, followed by an additional error field which as value has \google.rpc.Status<Status>\__ containing only code and message fields.

    • bigquery_destination_prefix (Optional[str]) – The BigQuery project or dataset location where the output is to be written to. If project is provided, a new dataset is created with name prediction_<model-display-name>_<job-create-time> where is made BigQuery-dataset-name compatible (for example, most special characters become underscores), and timestamp is in YYYY_MM_DDThh_mm_ss_sssZ “based on ISO-8601” format. In the dataset two tables will be created, predictions, and errors. If the Model has both [instance][google.cloud.aiplatform.v1.PredictSchemata.instance_schema_uri] and [prediction][google.cloud.aiplatform.v1.PredictSchemata.parameters_schema_uri] schemata defined then the tables have columns as follows: The predictions table contains instances for which the prediction succeeded, it has columns as per a concatenation of the Model’s instance and prediction schemata. The errors table contains rows for which the prediction has failed, it has instance columns, as per the instance schema, followed by a single “errors” column, which as values has [google.rpc.Status][google.rpc.Status] represented as a STRUCT, and containing only code and message.

    • model_parameters (Optional[Dict]) – The parameters that govern the predictions. The schema of the parameters may be specified via the Model’s parameters_schema_uri.

    • machine_type (Optional[str]) – The type of machine for running batch prediction on dedicated resources. Not specifying machine type will result in batch prediction job being run with automatic resources.

    • accelerator_type (Optional[str]) – The type of accelerator(s) that may be attached to the machine as per accelerator_count. Only used if machine_type is set.

    • accelerator_count (Optional[int]) – The number of accelerators to attach to the machine_type. Only used if machine_type is set.

    • starting_replica_count (Optional[int]) – The number of machine replicas used at the start of the batch operation. If not set, Vertex AI decides starting number, not greater than max_replica_count. Only used if machine_type is set.

    • max_replica_count (Optional[int]) – The maximum number of machine replicas the batch operation may be scaled to. Only used if machine_type is set. Default is 10.

    • generate_explanation (bool) – Optional. Generate explanation along with the batch prediction results. This will cause the batch prediction output to include explanations based on the prediction_format:

      * bigquery: output includes a column named explanation. The value

is a struct that conforms to the [aiplatform.gapic.Explanation] object.

>     * jsonl: The JSON objects on each line include an additional entry

>     keyed explanation. The value of the entry is a JSON object that
>     conforms to the [aiplatform.gapic.Explanation] object.


>     * csv: Generating explanations for CSV format is not supported.



* **explanation_metadata** (*aiplatform.explain.ExplanationMetadata*) – Optional. Explanation metadata configuration for this BatchPredictionJob.
Can be specified only if generate_explanation is set to True.

This value overrides the value of Model.explanation_metadata.
All fields of explanation_metadata are optional in the request. If
a field of the explanation_metadata object is not populated, the
corresponding field of the Model.explanation_metadata object is inherited.
For more details, see Ref docs <http://tinyurl.com/1igh60kt>



* **explanation_parameters** (*aiplatform.explain.ExplanationParameters*) – Optional. Parameters to configure explaining for Model’s predictions.
Can be specified only if generate_explanation is set to True.

This value overrides the value of Model.explanation_parameters.
All fields of explanation_parameters are optional in the request. If
a field of the explanation_parameters object is not populated, the
corresponding field of the Model.explanation_parameters object is inherited.
For more details, see Ref docs <http://tinyurl.com/1an4zake>



* **labels** (*Dict**[*[*str*](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)*, *[*str*](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)*]*) – Optional. The labels with user-defined metadata to organize your
BatchPredictionJobs. Label keys and values can be no longer than
64 characters (Unicode codepoints), can only contain lowercase
letters, numeric characters, underscores and dashes.
International characters are allowed. See [https://goo.gl/xmQnxf](https://goo.gl/xmQnxf)
for more information and examples of labels.


* **credentials** (*Optional**[**auth_credentials.Credentials**]*) – Custom credentials to use to create this batch prediction
job. Overrides credentials set in aiplatform.init.


* **encryption_spec_key_name** (*Optional**[*[*str*](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)*]*) – Optional. The Cloud KMS resource identifier of the customer
managed encryption key used to protect the job. Has the
form:
`projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key`.
The key needs to be in the same region as where the compute
resource is created.

If this is set, then all
resources created by the BatchPredictionJob will
be encrypted with the provided encryption key.

Overrides encryption_spec_key_name set in aiplatform.init.



* **sync** ([*bool*](https://python.readthedocs.io/en/latest/library/functions.html#bool)) – Whether to execute this method synchronously. If False, this method
will be executed in concurrent Future and any downstream object will
be immediately returned and synced when the Future has completed.


* **create_request_timeout** ([*float*](https://python.readthedocs.io/en/latest/library/functions.html#float)) – Optional. The timeout for the create request in seconds.


* **batch_size** ([*int*](https://python.readthedocs.io/en/latest/library/functions.html#int)) – Optional. The number of the records (e.g. instances) of the operation given in each batch
to a machine replica. Machine type, and size of a single record should be considered
when setting this parameter, higher value speeds up the batch operation’s execution,
but too high value will result in a whole batch not fitting in a machine’s memory,
and the whole operation will fail.
The default value is 64.


* **model_monitoring_objective_config** (*aiplatform.model_monitoring.ObjectiveConfig*) – Optional. The objective config for model monitoring. Passing this parameter enables
monitoring on the model associated with this batch prediction job.


* **model_monitoring_alert_config** (*aiplatform.model_monitoring.EmailAlertConfig*) – Optional. Configures how model monitoring alerts are sent to the user. Right now
only email alert is supported.


* **analysis_instance_schema_uri** ([*str*](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)) – Optional. Only applicable if model_monitoring_objective_config is also passed.
This parameter specifies the YAML schema file uri describing the format of a single
instance that you want Tensorflow Data Validation (TFDV) to
analyze. If this field is empty, all the feature data types are
inferred from predict_instance_schema_uri, meaning that TFDV
will use the data in the exact format as prediction request/response.
If there are any data type differences between predict instance
and TFDV instance, this field can be used to override the schema.
For models trained with Vertex AI, this field must be set as all the
fields in predict instance formatted as string.


* **service_account** ([*str*](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)) – Optional. Specifies the service account for workload run-as account.
Users submitting jobs must have act-as permission on this run-as account.

  • Returns

    Instantiated representation of the created batch prediction job.

  • Return type

    (jobs.BatchPredictionJob)

property create_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was created.

delete(sync: bool = True)

Deletes this Vertex AI resource. WARNING: This deletion is permanent.

  • Parameters

    sync (bool) – Whether to execute this deletion synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

property display_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Display name of this resource.

done()

Method indicating whether a job has completed.

  • Returns

    True if the job has completed.

property encryption_spec(: Optional[[google.cloud.aiplatform_v1.types.encryption_spec.EncryptionSpec](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.EncryptionSpec)_ )

Customer-managed encryption key options for this Vertex AI resource.

If this is set, then all resources created by this Vertex AI resource will be encrypted with the provided encryption key.

property end_time(: Optional[datetime.datetime )

Time when the Job resource entered the JOB_STATE_SUCCEEDED, JOB_STATE_FAILED, or JOB_STATE_CANCELLED state.

property error(: Optional[google.rpc.status_pb2.Status )

Detailed error info for this Job resource. Only populated when the Job’s state is JOB_STATE_FAILED or JOB_STATE_CANCELLED.

property gca_resource(: [proto.message.Message](https://proto-plus-python.readthedocs.io/en/latest/reference/message.html#proto.message.Message )

The underlying resource proto representation.

iter_outputs(bq_max_results: Optional[int] = 100)

Returns an Iterable object to traverse the output files, either a list of GCS Blobs or a BigQuery RowIterator depending on the output config set when the BatchPredictionJob was created.

  • Parameters

    bq_max_results – Optional[int] = 100 Limit on rows to retrieve from prediction table in BigQuery dataset. Only used when retrieving predictions from a bigquery_destination_prefix. Default is 100.

  • Returns

    Either a list of GCS Blob objects within the prediction output directory or an iterable BigQuery RowIterator with predictions.

  • Return type

    Union[Iterable[storage.Blob], Iterable[bigquery.table.RowIterator]]

  • Raises

    • RuntimeError – If BatchPredictionJob is in a JobState other than SUCCEEDED, since outputs cannot be retrieved until the Job has finished.

    • NotImplementedError – If BatchPredictionJob succeeded and output_info does not have a GCS or BQ output provided.

property labels(: Dict[str, str )

User-defined labels containing metadata about this resource.

Read more about labels at https://goo.gl/xmQnxf

classmethod list(filter: Optional[str] = None, order_by: Optional[str] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

List all instances of this Job Resource.

Example Usage:

aiplatform.BatchPredictionJobs.list(

filter=’state=”JOB_STATE_SUCCEEDED” AND display_name=”my_job”’,

)

  • Parameters

    • filter (str) – Optional. An expression for filtering the results of the request. For field names both snake_case and camelCase are supported.

    • order_by (str) – Optional. A comma-separated list of fields to order by, sorted in ascending order. Use “desc” after a field name for descending. Supported fields: display_name, create_time, update_time

    • project (str) – Optional. Project to retrieve list from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to retrieve list from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to retrieve list. Overrides credentials set in aiplatform.init.

  • Returns

    List[VertexAiResourceNoun] - A list of Job resource objects.

property name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Name of this resource.

property output_info(: Optional[[google.cloud.aiplatform_v1.types.batch_prediction_job.BatchPredictionJob.OutputInfo](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.BatchPredictionJob.OutputInfo)_ )

Information describing the output of this job, including output location into which prediction output is written.

This is only available for batch prediction jobs that have run successfully.

property partial_failures(: Optional[Sequence[google.rpc.status_pb2.Status] )

Partial failures encountered. For example, single files that can’t be read. This field never exceeds 20 entries. Status details fields contain standard GCP error details.

property resource_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Full qualified resource name.

property start_time(: Optional[datetime.datetime )

Time when the Job resource entered the JOB_STATE_RUNNING for the first time.

property state(: [google.cloud.aiplatform_v1.types.job_state.JobState](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.JobState_ )

Fetch Job again and return the current JobState.

  • Returns

    Enum that describes the state of a Vertex AI job.

  • Return type

    state (job_state.JobState)

classmethod submit(*, job_display_name: Optional[str] = None, model_name: Union[str, google.cloud.aiplatform.models.Model], instances_format: str = 'jsonl', predictions_format: str = 'jsonl', gcs_source: Optional[Union[str, Sequence[str]]] = None, bigquery_source: Optional[str] = None, gcs_destination_prefix: Optional[str] = None, bigquery_destination_prefix: Optional[str] = None, model_parameters: Optional[Dict] = None, machine_type: Optional[str] = None, accelerator_type: Optional[str] = None, accelerator_count: Optional[int] = None, starting_replica_count: Optional[int] = None, max_replica_count: Optional[int] = None, generate_explanation: Optional[bool] = False, explanation_metadata: Optional[google.cloud.aiplatform_v1.types.explanation_metadata.ExplanationMetadata] = None, explanation_parameters: Optional[google.cloud.aiplatform_v1.types.explanation.ExplanationParameters] = None, labels: Optional[Dict[str, str]] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None, encryption_spec_key_name: Optional[str] = None, create_request_timeout: Optional[float] = None, batch_size: Optional[int] = None, model_monitoring_objective_config: Optional[google.cloud.aiplatform.model_monitoring.objective.ObjectiveConfig] = None, model_monitoring_alert_config: Optional[google.cloud.aiplatform.model_monitoring.alert.AlertConfig] = None, analysis_instance_schema_uri: Optional[str] = None, service_account: Optional[str] = None)

Sumbit a batch prediction job (not waiting for completion).

  • Parameters

    • job_display_name (str) – Required. The user-defined name of the BatchPredictionJob. The name can be up to 128 characters long and can be consist of any UTF-8 characters.

    • model_name (Union[str, **aiplatform.Model]) – Required. A fully-qualified model resource name or model ID. Example: “projects/123/locations/us-central1/models/456” or “456” when project and location are initialized or passed. May optionally contain a version ID or alias in {model_name}@{version} form.

      Or an instance of aiplatform.Model.

    • instances_format (str) – Required. The format in which instances are provided. Must be one of the formats listed in Model.supported_input_storage_formats. Default is “jsonl” when using gcs_source. If a bigquery_source is provided, this is overridden to “bigquery”.

    • predictions_format (str) – Required. The format in which Vertex AI outputs the predictions, must be one of the formats specified in Model.supported_output_storage_formats. Default is “jsonl” when using gcs_destination_prefix. If a bigquery_destination_prefix is provided, this is overridden to “bigquery”.

    • gcs_source (Optional[Sequence[str]]) – Google Cloud Storage URI(-s) to your instances to run batch prediction on. They must match instances_format.

    • bigquery_source (Optional[str]) – BigQuery URI to a table, up to 2000 characters long. For example: bq://projectId.bqDatasetId.bqTableId

    • gcs_destination_prefix (Optional[str]) – The Google Cloud Storage location of the directory where the output is to be written to. In the given directory a new directory is created. Its name is prediction-<model-display-name>-<job-create-time>, where timestamp is in YYYY-MM-DDThh:mm:ss.sssZ ISO-8601 format. Inside of it files predictions_0001.<extension>, predictions_0002.<extension>, …, predictions_N.<extension> are created where <extension> depends on chosen predictions_format, and N may equal 0001 and depends on the total number of successfully predicted instances. If the Model has both instance and prediction schemata defined then each such file contains predictions as per the predictions_format. If prediction for any instance failed (partially or completely), then an additional errors_0001.<extension>, errors_0002.<extension>,…, errors_N.<extension> files are created (N depends on total number of failed predictions). These files contain the failed instances, as per their schema, followed by an additional error field which as value has \google.rpc.Status<Status>\__ containing only code and message fields.

    • bigquery_destination_prefix (Optional[str]) – The BigQuery project or dataset location where the output is to be written to. If project is provided, a new dataset is created with name prediction_<model-display-name>_<job-create-time> where is made BigQuery-dataset-name compatible (for example, most special characters become underscores), and timestamp is in YYYY_MM_DDThh_mm_ss_sssZ “based on ISO-8601” format. In the dataset two tables will be created, predictions, and errors. If the Model has both [instance][google.cloud.aiplatform.v1.PredictSchemata.instance_schema_uri] and [prediction][google.cloud.aiplatform.v1.PredictSchemata.parameters_schema_uri] schemata defined then the tables have columns as follows: The predictions table contains instances for which the prediction succeeded, it has columns as per a concatenation of the Model’s instance and prediction schemata. The errors table contains rows for which the prediction has failed, it has instance columns, as per the instance schema, followed by a single “errors” column, which as values has [google.rpc.Status][google.rpc.Status] represented as a STRUCT, and containing only code and message.

    • model_parameters (Optional[Dict]) – The parameters that govern the predictions. The schema of the parameters may be specified via the Model’s parameters_schema_uri.

    • machine_type (Optional[str]) – The type of machine for running batch prediction on dedicated resources. Not specifying machine type will result in batch prediction job being run with automatic resources.

    • accelerator_type (Optional[str]) – The type of accelerator(s) that may be attached to the machine as per accelerator_count. Only used if machine_type is set.

    • accelerator_count (Optional[int]) – The number of accelerators to attach to the machine_type. Only used if machine_type is set.

    • starting_replica_count (Optional[int]) – The number of machine replicas used at the start of the batch operation. If not set, Vertex AI decides starting number, not greater than max_replica_count. Only used if machine_type is set.

    • max_replica_count (Optional[int]) – The maximum number of machine replicas the batch operation may be scaled to. Only used if machine_type is set. Default is 10.

    • generate_explanation (bool) – Optional. Generate explanation along with the batch prediction results. This will cause the batch prediction output to include explanations based on the prediction_format:

      * bigquery: output includes a column named explanation. The value

is a struct that conforms to the [aiplatform.gapic.Explanation] object.

>     * jsonl: The JSON objects on each line include an additional entry

>     keyed explanation. The value of the entry is a JSON object that
>     conforms to the [aiplatform.gapic.Explanation] object.


>     * csv: Generating explanations for CSV format is not supported.



* **explanation_metadata** (*aiplatform.explain.ExplanationMetadata*) – Optional. Explanation metadata configuration for this BatchPredictionJob.
Can be specified only if generate_explanation is set to True.

This value overrides the value of Model.explanation_metadata.
All fields of explanation_metadata are optional in the request. If
a field of the explanation_metadata object is not populated, the
corresponding field of the Model.explanation_metadata object is inherited.
For more details, see Ref docs <http://tinyurl.com/1igh60kt>



* **explanation_parameters** (*aiplatform.explain.ExplanationParameters*) – Optional. Parameters to configure explaining for Model’s predictions.
Can be specified only if generate_explanation is set to True.

This value overrides the value of Model.explanation_parameters.
All fields of explanation_parameters are optional in the request. If
a field of the explanation_parameters object is not populated, the
corresponding field of the Model.explanation_parameters object is inherited.
For more details, see Ref docs <http://tinyurl.com/1an4zake>



* **labels** (*Dict**[*[*str*](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)*, *[*str*](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)*]*) – Optional. The labels with user-defined metadata to organize your
BatchPredictionJobs. Label keys and values can be no longer than
64 characters (Unicode codepoints), can only contain lowercase
letters, numeric characters, underscores and dashes.
International characters are allowed. See [https://goo.gl/xmQnxf](https://goo.gl/xmQnxf)
for more information and examples of labels.


* **credentials** (*Optional**[**auth_credentials.Credentials**]*) – Custom credentials to use to create this batch prediction
job. Overrides credentials set in aiplatform.init.


* **encryption_spec_key_name** (*Optional**[*[*str*](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)*]*) – Optional. The Cloud KMS resource identifier of the customer
managed encryption key used to protect the job. Has the
form:
`projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key`.
The key needs to be in the same region as where the compute
resource is created.

If this is set, then all
resources created by the BatchPredictionJob will
be encrypted with the provided encryption key.

Overrides encryption_spec_key_name set in aiplatform.init.



* **create_request_timeout** ([*float*](https://python.readthedocs.io/en/latest/library/functions.html#float)) – Optional. The timeout for the create request in seconds.


* **batch_size** ([*int*](https://python.readthedocs.io/en/latest/library/functions.html#int)) – Optional. The number of the records (e.g. instances) of the operation given in each batch
to a machine replica. Machine type, and size of a single record should be considered
when setting this parameter, higher value speeds up the batch operation’s execution,
but too high value will result in a whole batch not fitting in a machine’s memory,
and the whole operation will fail.
The default value is 64.


* **model_monitoring_objective_config** (*aiplatform.model_monitoring.ObjectiveConfig*) – Optional. The objective config for model monitoring. Passing this parameter enables
monitoring on the model associated with this batch prediction job.


* **model_monitoring_alert_config** (*aiplatform.model_monitoring.EmailAlertConfig*) – Optional. Configures how model monitoring alerts are sent to the user. Right now
only email alert is supported.


* **analysis_instance_schema_uri** ([*str*](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)) – Optional. Only applicable if model_monitoring_objective_config is also passed.
This parameter specifies the YAML schema file uri describing the format of a single
instance that you want Tensorflow Data Validation (TFDV) to
analyze. If this field is empty, all the feature data types are
inferred from predict_instance_schema_uri, meaning that TFDV
will use the data in the exact format as prediction request/response.
If there are any data type differences between predict instance
and TFDV instance, this field can be used to override the schema.
For models trained with Vertex AI, this field must be set as all the
fields in predict instance formatted as string.


* **service_account** ([*str*](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)) – Optional. Specifies the service account for workload run-as account.
Users submitting jobs must have act-as permission on this run-as account.

  • Returns

    Instantiated representation of the created batch prediction job.

  • Return type

    (jobs.BatchPredictionJob)

to_dict()

Returns the resource proto as a dictionary.

property update_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was last updated.

wait()

Helper method that blocks until all futures are complete.

wait_for_completion()

Waits for job to complete.

wait_for_resource_creation()

Waits until resource has been created.

class google.cloud.aiplatform.CustomContainerTrainingJob(display_name: str, container_uri: str, command: Optional[Sequence[str]] = None, model_serving_container_image_uri: Optional[str] = None, model_serving_container_predict_route: Optional[str] = None, model_serving_container_health_route: Optional[str] = None, model_serving_container_command: Optional[Sequence[str]] = None, model_serving_container_args: Optional[Sequence[str]] = None, model_serving_container_environment_variables: Optional[Dict[str, str]] = None, model_serving_container_ports: Optional[Sequence[int]] = None, model_description: Optional[str] = None, model_instance_schema_uri: Optional[str] = None, model_parameters_schema_uri: Optional[str] = None, model_prediction_schema_uri: Optional[str] = None, explanation_metadata: Optional[google.cloud.aiplatform_v1.types.explanation_metadata.ExplanationMetadata] = None, explanation_parameters: Optional[google.cloud.aiplatform_v1.types.explanation.ExplanationParameters] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None, labels: Optional[Dict[str, str]] = None, training_encryption_spec_key_name: Optional[str] = None, model_encryption_spec_key_name: Optional[str] = None, staging_bucket: Optional[str] = None)

Bases: google.cloud.aiplatform.training_jobs._CustomTrainingJob

Class to launch a Custom Training Job in Vertex AI using a Container.

Constructs a Custom Container Training Job.

job = aiplatform.CustomContainerTrainingJob(

display_name=’test-train’,
container_uri=’gcr.io/my_project_id/my_image_name:tag’,
command=[‘python3’, ‘run_script.py’]
model_serving_container_image_uri=’gcr.io/my-trainer/serving:1’,
model_serving_container_predict_route=’predict’,
model_serving_container_health_route=’metadata,
labels={‘key’: ‘value’},

)

Usage with Dataset:

ds = aiplatform.TabularDataset(

‘projects/my-project/locations/us-central1/datasets/12345’)

job.run(

ds,
replica_count=1,
model_display_name=’my-trained-model’,
model_labels={‘key’: ‘value’},

)

Usage without Dataset:

job.run(replica_count=1, model_display_name=’my-trained-model)

To ensure your model gets saved in Vertex AI, write your saved model to os.environ[“AIP_MODEL_DIR”] in your provided training script.

  • Parameters

    • display_name (str) – Required. The user-defined name of this TrainingPipeline.

    • container_uri (str) – Required: Uri of the training container image in the GCR.

    • command (Sequence[str]) – The command to be invoked when the container is started. It overrides the entrypoint instruction in Dockerfile when provided

    • model_serving_container_image_uri (str) – If the training produces a managed Vertex AI Model, the URI of the Model serving container suitable for serving the model produced by the training script.

    • model_serving_container_predict_route (str) – If the training produces a managed Vertex AI Model, An HTTP path to send prediction requests to the container, and which must be supported by it. If not specified a default HTTP path will be used by Vertex AI.

    • model_serving_container_health_route (str) – If the training produces a managed Vertex AI Model, an HTTP path to send health check requests to the container, and which must be supported by it. If not specified a standard HTTP path will be used by AI Platform.

    • model_serving_container_command (Sequence[str]) – The command with which the container is run. Not executed within a shell. The Docker image’s ENTRYPOINT is used if this is not provided. Variable references $(VAR_NAME) are expanded using the container’s environment. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not.

    • model_serving_container_args (Sequence[str]) – The arguments to the command. The Docker image’s CMD is used if this is not provided. Variable references $(VAR_NAME) are expanded using the container’s environment. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not.

    • model_serving_container_environment_variables (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – The environment variables that are to be present in the container. Should be a dictionary where keys are environment variable names and values are environment variable values for those names.

    • model_serving_container_ports (Sequence[int]) – Declaration of ports that are exposed by the container. This field is primarily informational, it gives Vertex AI information about the network connections the container uses. Listing or not a port here has no impact on whether the port is actually exposed, any port listening on the default “0.0.0.0” address inside a container will be accessible from the network.

    • model_description (str) – The description of the Model.

    • model_instance_schema_uri (str) – Optional. Points to a YAML file stored on Google Cloud Storage describing the format of a single instance, which are used in PredictRequest.instances, ExplainRequest.instances and BatchPredictionJob.input_config. The schema is defined as an OpenAPI 3.0.2 Schema Object. AutoML Models always have this field populated by AI Platform. Note: The URI given on output will be immutable and probably different, including the URI scheme, than the one given on input. The output URI will point to a location where the user only has a read access.

    • model_parameters_schema_uri (str) – Optional. Points to a YAML file stored on Google Cloud Storage describing the parameters of prediction and explanation via PredictRequest.parameters, ExplainRequest.parameters and BatchPredictionJob.model_parameters. The schema is defined as an OpenAPI 3.0.2 Schema Object. AutoML Models always have this field populated by AI Platform, if no parameters are supported it is set to an empty string. Note: The URI given on output will be immutable and probably different, including the URI scheme, than the one given on input. The output URI will point to a location where the user only has a read access.

    • model_prediction_schema_uri (str) – Optional. Points to a YAML file stored on Google Cloud Storage describing the format of a single prediction produced by this Model, which are returned via PredictResponse.predictions, ExplainResponse.explanations, and BatchPredictionJob.output_config. The schema is defined as an OpenAPI 3.0.2 Schema Object. AutoML Models always have this field populated by AI Platform. Note: The URI given on output will be immutable and probably different, including the URI scheme, than the one given on input. The output URI will point to a location where the user only has a read access.

    • explanation_metadata (explain.ExplanationMetadata) – Optional. Metadata describing the Model’s input and output for explanation. explanation_metadata is optional while explanation_parameters must be specified when used. For more details, see Ref docs http://tinyurl.com/1igh60kt

    • explanation_parameters (explain.ExplanationParameters) – Optional. Parameters to configure explaining for Model’s predictions. For more details, see Ref docs http://tinyurl.com/1an4zake

    • project (str) – Project to run training in. Overrides project set in aiplatform.init.

    • location (str) – Location to run training in. Overrides location set in aiplatform.init.

    • credentials (auth_credentials.Credentials) – Custom credentials to use to run call training service. Overrides credentials set in aiplatform.init.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The labels with user-defined metadata to organize TrainingPipelines. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information and examples of labels.

    • training_encryption_spec_key_name (Optional[str]) – Optional. The Cloud KMS resource identifier of the customer managed encryption key used to protect the training pipeline. Has the form: projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key. The key needs to be in the same region as where the compute resource is created.

      If set, this TrainingPipeline will be secured by this key.

      Note: Model trained by this TrainingPipeline is also secured by this key if model_to_upload is not set separately.

      Overrides encryption_spec_key_name set in aiplatform.init.

    • model_encryption_spec_key_name (Optional[str]) – Optional. The Cloud KMS resource identifier of the customer managed encryption key used to protect the model. Has the form: projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key. The key needs to be in the same region as where the compute resource is created.

      If set, the trained Model will be secured by this key.

      Overrides encryption_spec_key_name set in aiplatform.init.

    • staging_bucket (str) – Bucket used to stage source and training artifacts. Overrides staging_bucket set in aiplatform.init.

cancel()

Starts asynchronous cancellation on the TrainingJob. The server makes a best effort to cancel the job, but success is not guaranteed. On successful cancellation, the TrainingJob is not deleted; instead it becomes a job with state set to CANCELLED.

  • Raises

    RuntimeError – If this TrainingJob has not started running.

client_class()

alias of google.cloud.aiplatform.utils.PipelineClientWithOverride

property create_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was created.

delete(sync: bool = True)

Deletes this Vertex AI resource. WARNING: This deletion is permanent.

  • Parameters

    sync (bool) – Whether to execute this deletion synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

property display_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Display name of this resource.

done()

Method indicating whether a job has completed.

  • Returns

    True if the job has completed.

property encryption_spec(: Optional[[google.cloud.aiplatform_v1.types.encryption_spec.EncryptionSpec](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.EncryptionSpec)_ )

Customer-managed encryption key options for this Vertex AI resource.

If this is set, then all resources created by this Vertex AI resource will be encrypted with the provided encryption key.

property end_time(: Optional[datetime.datetime )

Time when the TrainingJob resource entered the PIPELINE_STATE_SUCCEEDED, PIPELINE_STATE_FAILED, PIPELINE_STATE_CANCELLED state.

property error(: Optional[google.rpc.status_pb2.Status )

Detailed error info for this TrainingJob resource. Only populated when the TrainingJob’s state is PIPELINE_STATE_FAILED or PIPELINE_STATE_CANCELLED.

property gca_resource(: [proto.message.Message](https://proto-plus-python.readthedocs.io/en/latest/reference/message.html#proto.message.Message )

The underlying resource proto representation.

classmethod get(resource_name: str, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Get Training Job for the given resource_name.

  • Parameters

    • resource_name (str) – Required. A fully-qualified resource name or ID.

    • project (str) – Optional project to retrieve training job from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional location to retrieve training job from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Custom credentials to use to upload this model. Overrides credentials set in aiplatform.init.

  • Raises

    ValueError – If the retrieved training job’s training task definition doesn’t match the custom training task definition.

  • Returns

    A Vertex AI Training Job

get_model(sync=True)

Vertex AI Model produced by this training, if one was produced.

  • Parameters

    sync (bool) – Whether to execute this method synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

  • Returns

    Vertex AI Model produced by this training

  • Return type

    model

  • Raises

    RuntimeError – If training failed or if a model was not produced by this training.

property has_failed(: [bool](https://python.readthedocs.io/en/latest/library/functions.html#bool )

Returns True if training has failed.

False otherwise.

property labels(: Dict[str, str )

User-defined labels containing metadata about this resource.

Read more about labels at https://goo.gl/xmQnxf

classmethod list(filter: Optional[str] = None, order_by: Optional[str] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

List all instances of this TrainingJob resource.

Example Usage:

aiplatform.CustomTrainingJob.list(

filter=’display_name=”experiment_a27”’,
order_by=’create_time desc’

)

  • Parameters

    • filter (str) – Optional. An expression for filtering the results of the request. For field names both snake_case and camelCase are supported.

    • order_by (str) – Optional. A comma-separated list of fields to order by, sorted in ascending order. Use “desc” after a field name for descending. Supported fields: display_name, create_time, update_time

    • project (str) – Optional. Project to retrieve list from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to retrieve list from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to retrieve list. Overrides credentials set in aiplatform.init.

  • Returns

    List[VertexAiResourceNoun] - A list of TrainingJob resource objects

property name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Name of this resource.

property network(: Optional[str )

The full name of the Google Compute Engine network to which this CustomTrainingJob should be peered.

Takes the format projects/{project}/global/networks/{network}. Where {project} is a project number, as in 12345, and {network} is a network name.

Private services access must already be configured for the network. If left unspecified, the CustomTrainingJob is not peered with any network.

property resource_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Full qualified resource name.

run(dataset: Optional[Union[google.cloud.aiplatform.datasets.image_dataset.ImageDataset, google.cloud.aiplatform.datasets.tabular_dataset.TabularDataset, google.cloud.aiplatform.datasets.text_dataset.TextDataset, google.cloud.aiplatform.datasets.video_dataset.VideoDataset]] = None, annotation_schema_uri: Optional[str] = None, model_display_name: Optional[str] = None, model_labels: Optional[Dict[str, str]] = None, model_id: Optional[str] = None, parent_model: Optional[str] = None, is_default_version: Optional[bool] = True, model_version_aliases: Optional[Sequence[str]] = None, model_version_description: Optional[str] = None, base_output_dir: Optional[str] = None, service_account: Optional[str] = None, network: Optional[str] = None, bigquery_destination: Optional[str] = None, args: Optional[List[Union[str, float, int]]] = None, environment_variables: Optional[Dict[str, str]] = None, replica_count: int = 1, machine_type: str = 'n1-standard-4', accelerator_type: str = 'ACCELERATOR_TYPE_UNSPECIFIED', accelerator_count: int = 0, boot_disk_type: str = 'pd-ssd', boot_disk_size_gb: int = 100, reduction_server_replica_count: int = 0, reduction_server_machine_type: Optional[str] = None, reduction_server_container_uri: Optional[str] = None, training_fraction_split: Optional[float] = None, validation_fraction_split: Optional[float] = None, test_fraction_split: Optional[float] = None, training_filter_split: Optional[str] = None, validation_filter_split: Optional[str] = None, test_filter_split: Optional[str] = None, predefined_split_column_name: Optional[str] = None, timestamp_split_column_name: Optional[str] = None, timeout: Optional[int] = None, restart_job_on_worker_restart: bool = False, enable_web_access: bool = False, enable_dashboard_access: bool = False, tensorboard: Optional[str] = None, sync=True, create_request_timeout: Optional[float] = None, disable_retries: bool = False, persistent_resource_id: Optional[str] = None, tpu_topology: Optional[str] = None)

Runs the custom training job.

Distributed Training Support: If replica count = 1 then one chief replica will be provisioned. If replica_count > 1 the remainder will be provisioned as a worker replica pool. ie: replica_count = 10 will result in 1 chief and 9 workers All replicas have same machine_type, accelerator_type, and accelerator_count

If training on a Vertex AI dataset, you can use one of the following split configurations:

Data fraction splits:
Any of `training_fraction_split`, `validation_fraction_split` and
`test_fraction_split` may optionally be provided, they must sum to up to 1. If
the provided ones sum to less than 1, the remainder is assigned to sets as
decided by Vertex AI. If none of the fractions are set, by default roughly 80%
of data will be used for training, 10% for validation, and 10% for test.

Data filter splits:
Assigns input data to training, validation, and test sets
based on the given filters, data pieces not matched by any
filter are ignored. Currently only supported for Datasets
containing DataItems.
If any of the filters in this message are to match nothing, then
they can be set as ‘-’ (the minus sign).
If using filter splits, all of `training_filter_split`, `validation_filter_split` and
`test_filter_split` must be provided.
Supported only for unstructured Datasets.

Predefined splits:
Assigns input data to training, validation, and test sets based on the value of a provided key.
If using predefined splits, `predefined_split_column_name` must be provided.
Supported only for tabular Datasets.

Timestamp splits:
Assigns input data to training, validation, and test sets
based on a provided timestamps. The youngest data pieces are
assigned to training set, next to validation set, and the oldest
to the test set.
Supported only for tabular Datasets.

  • Parameters

    • dataset (Union[datasets.ImageDataset,datasets.TabularDataset,datasets.TextDataset,datasets.VideoDataset]) – Vertex AI to fit this training against. Custom training script should retrieve datasets through passed in environment variables uris:

      os.environ[“AIP_TRAINING_DATA_URI”] os.environ[“AIP_VALIDATION_DATA_URI”] os.environ[“AIP_TEST_DATA_URI”]

      Additionally the dataset format is passed in as:

      os.environ[“AIP_DATA_FORMAT”]

    • annotation_schema_uri (str) – Google Cloud Storage URI points to a YAML file describing annotation schema. The schema is defined as an OpenAPI 3.0.2 Schema Object The schema files that can be used here are found in gs://google-cloud-aiplatform/schema/dataset/annotation/, note that the chosen schema must be consistent with metadata of the Dataset specified by dataset_id.

      Only Annotations that both match this schema and belong to DataItems not ignored by the split method are used in respectively training, validation or test role, depending on the role of the DataItem they are on.

      When used in conjunction with annotations_filter, the Annotations used for training are filtered by both annotations_filter and annotation_schema_uri.

    • model_display_name (str) – If the script produces a managed Vertex AI Model. The display name of the Model. The name can be up to 128 characters long and can be consist of any UTF-8 characters.

      If not provided upon creation, the job’s display_name is used.

    • model_labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The labels with user-defined metadata to organize your Models. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information and examples of labels.

    • model_id (str) – Optional. The ID to use for the Model produced by this job, which will become the final component of the model resource name. This value may be up to 63 characters, and valid characters are [a-z0-9_-]. The first character cannot be a number or hyphen.

    • parent_model (str) – Optional. The resource name or model ID of an existing model. The new model uploaded by this job will be a version of parent_model.

      Only set this field when training a new version of an existing model.

    • is_default_version (bool) – Optional. When set to True, the newly uploaded model version will automatically have alias “default” included. Subsequent uses of the model produced by this job without a version specified will use this “default” version.

      When set to False, the “default” alias will not be moved. Actions targeting the model version produced by this job will need to specifically reference this version by ID or alias.

      New model uploads, i.e. version 1, will always be “default” aliased.

    • model_version_aliases (Sequence[str]) – Optional. User provided version aliases so that the model version uploaded by this job can be referenced via alias instead of auto-generated version ID. A default version alias will be created for the first version of the model.

      The format is [a-z][a-zA-Z0-9-]{0,126}[a-z0-9]

    • model_version_description (str) – Optional. The description of the model version being uploaded by this job.

    • base_output_dir (str) – GCS output directory of job. If not provided a timestamped directory in the staging directory will be used.

      Vertex AI sets the following environment variables when it runs your training code:

      • AIP_MODEL_DIR: a Cloud Storage URI of a directory intended for saving model artifacts, i.e. <base_output_dir>/model/

      • AIP_CHECKPOINT_DIR: a Cloud Storage URI of a directory intended for saving checkpoints, i.e. <base_output_dir>/checkpoints/

      • AIP_TENSORBOARD_LOG_DIR: a Cloud Storage URI of a directory intended for saving TensorBoard logs, i.e. <base_output_dir>/logs/

    • service_account (str) – Specifies the service account for workload run-as account. Users submitting jobs must have act-as permission on this run-as account.

    • network (str) – The full name of the Compute Engine network to which the job should be peered. For example, projects/12345/global/networks/myVPC. Private services access must already be configured for the network. If left unspecified, the network set in aiplatform.init will be used. Otherwise, the job is not peered with any network.

    • bigquery_destination (str) – Provide this field if dataset is a BigQuery dataset. The BigQuery project location where the training data is to be written to. In the given project a new dataset is created with name dataset_<dataset-id>_<annotation-type>_<timestamp-of-training-call> where timestamp is in YYYY_MM_DDThh_mm_ss_sssZ format. All training input data will be written into that dataset. In the dataset three tables will be created, training, validation and test.

      • AIP_DATA_FORMAT = “bigquery”.

      • AIP_TRAINING_DATA_URI =”bigquery_destination.dataset_*.training”

      • AIP_VALIDATION_DATA_URI = “bigquery_destination.dataset_*.validation”

      • AIP_TEST_DATA_URI = “bigquery_destination.dataset_*.test”

    • args (List[Unions[str, *[int](https://python.readthedocs.io/en/latest/library/functions.html#int), [float](https://python.readthedocs.io/en/latest/library/functions.html#float)]*]) – Command line arguments to be passed to the Python script.

    • environment_variables (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Environment variables to be passed to the container. Should be a dictionary where keys are environment variable names and values are environment variable values for those names. At most 10 environment variables can be specified. The Name of the environment variable must be unique.

      environment_variables = {

      ‘MY_KEY’: ‘MY_VALUE’

      }

    • replica_count (int) – The number of worker replicas. If replica count = 1 then one chief replica will be provisioned. If replica_count > 1 the remainder will be provisioned as a worker replica pool.

    • machine_type (str) – The type of machine to use for training.

    • accelerator_type (str) – Hardware accelerator type. One of ACCELERATOR_TYPE_UNSPECIFIED, NVIDIA_TESLA_K80, NVIDIA_TESLA_P100, NVIDIA_TESLA_V100, NVIDIA_TESLA_P4, NVIDIA_TESLA_T4

    • accelerator_count (int) – The number of accelerators to attach to a worker replica.

    • boot_disk_type (str) – Type of the boot disk, default is pd-ssd. Valid values: pd-ssd (Persistent Disk Solid State Drive) or pd-standard (Persistent Disk Hard Disk Drive).

    • boot_disk_size_gb (int) – Size in GB of the boot disk, default is 100GB. boot disk size must be within the range of [100, 64000].

    • reduction_server_replica_count (int) – The number of reduction server replicas, default is 0.

    • reduction_server_machine_type (str) – Optional. The type of machine to use for reduction server.

    • reduction_server_container_uri (str) – Optional. The Uri of the reduction server container image. See details: https://cloud.google.com/vertex-ai/docs/training/distributed-training#reduce_training_time_with_reduction_server

    • training_fraction_split (float) – Optional. The fraction of the input data that is to be used to train the Model. This is ignored if Dataset is not provided.

    • validation_fraction_split (float) – Optional. The fraction of the input data that is to be used to validate the Model. This is ignored if Dataset is not provided.

    • test_fraction_split (float) – Optional. The fraction of the input data that is to be used to evaluate the Model. This is ignored if Dataset is not provided.

    • training_filter_split (str) – Optional. A filter on DataItems of the Dataset. DataItems that match this filter are used to train the Model. A filter with same syntax as the one used in DatasetService.ListDataItems may be used. If a single DataItem is matched by more than one of the FilterSplit filters, then it is assigned to the first set that applies to it in the training, validation, test order. This is ignored if Dataset is not provided.

    • validation_filter_split (str) – Optional. A filter on DataItems of the Dataset. DataItems that match this filter are used to validate the Model. A filter with same syntax as the one used in DatasetService.ListDataItems may be used. If a single DataItem is matched by more than one of the FilterSplit filters, then it is assigned to the first set that applies to it in the training, validation, test order. This is ignored if Dataset is not provided.

    • test_filter_split (str) – Optional. A filter on DataItems of the Dataset. DataItems that match this filter are used to test the Model. A filter with same syntax as the one used in DatasetService.ListDataItems may be used. If a single DataItem is matched by more than one of the FilterSplit filters, then it is assigned to the first set that applies to it in the training, validation, test order. This is ignored if Dataset is not provided.

    • predefined_split_column_name (str) – Optional. The key is a name of one of the Dataset’s data columns. The value of the key (either the label’s value or value in the column) must be one of {training, validation, test}, and it defines to which set the given piece of data is assigned. If for a piece of data the key is not present or has an invalid value, that piece is ignored by the pipeline.

      Supported only for tabular and time series Datasets.

    • timestamp_split_column_name (str) – Optional. The key is a name of one of the Dataset’s data columns. The value of the key values of the key (the values in the column) must be in RFC 3339 date-time format, where time-offset = “Z” (e.g. 1985-04-12T23:20:50.52Z). If for a piece of data the key is not present or has an invalid value, that piece is ignored by the pipeline.

      Supported only for tabular and time series Datasets.

    • timeout (int) – The maximum job running time in seconds. The default is 7 days.

    • restart_job_on_worker_restart (bool) – Restarts the entire CustomJob if a worker gets restarted. This feature can be used by distributed training jobs that are not resilient to workers leaving and joining a job.

    • enable_web_access (bool) – Whether you want Vertex AI to enable interactive shell access to training containers. https://cloud.google.com/vertex-ai/docs/training/monitor-debug-interactive-shell

    • enable_dashboard_access (bool) – Whether you want Vertex AI to enable access to the customized dashboard to training containers.

    • tensorboard (str) – Optional. The name of a Vertex AI [Tensorboard][google.cloud.aiplatform.v1beta1.Tensorboard] resource to which this CustomJob will upload Tensorboard logs. Format: projects/{project}/locations/{location}/tensorboards/{tensorboard}

      The training script should write Tensorboard to following Vertex AI environment variable:

      AIP_TENSORBOARD_LOG_DIR

      service_account is required with provided tensorboard. For more information on configuring your service account please visit: https://cloud.google.com/vertex-ai/docs/experiments/tensorboard-training

    • sync (bool) – Whether to execute this method synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

    • create_request_timeout (float) – Optional. The timeout for the create request in seconds.

    • disable_retries (bool) – Indicates if the job should retry for internal errors after the job starts running. If True, overrides restart_job_on_worker_restart to False.

    • persistent_resource_id (str) – Optional. The ID of the PersistentResource in the same Project and Location. If this is specified, the job will be run on existing machines held by the PersistentResource instead of on-demand short-live machines. The network, CMEK, and node pool configs on the job should be consistent with those on the PersistentResource, otherwise, the job will be rejected.

    • tpu_topology (str) – Optional. Specifies the tpu topology to be used for TPU training job. This field is required for TPU v5 versions. For details on the TPU topology, refer to https://cloud.google.com/tpu/docs/v5e#tpu-v5e-config. The topology must be a supported value for the TPU machine type.

  • Returns

    The trained Vertex AI Model resource or None if training did not

      produce a Vertex AI Model.

  • Return type

    model

  • Raises

    RuntimeError – If Training job has already been run, staging_bucket has not been set, or model_display_name was provided but required arguments were not provided in constructor.

property start_time(: Optional[datetime.datetime )

Time when the TrainingJob entered the PIPELINE_STATE_RUNNING for the first time.

property state(: Optional[[google.cloud.aiplatform_v1.types.pipeline_state.PipelineState](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.PipelineState)_ )

Current training state.

submit(dataset: Optional[Union[google.cloud.aiplatform.datasets.image_dataset.ImageDataset, google.cloud.aiplatform.datasets.tabular_dataset.TabularDataset, google.cloud.aiplatform.datasets.text_dataset.TextDataset, google.cloud.aiplatform.datasets.video_dataset.VideoDataset]] = None, annotation_schema_uri: Optional[str] = None, model_display_name: Optional[str] = None, model_labels: Optional[Dict[str, str]] = None, model_id: Optional[str] = None, parent_model: Optional[str] = None, is_default_version: Optional[bool] = True, model_version_aliases: Optional[Sequence[str]] = None, model_version_description: Optional[str] = None, base_output_dir: Optional[str] = None, service_account: Optional[str] = None, network: Optional[str] = None, bigquery_destination: Optional[str] = None, args: Optional[List[Union[str, float, int]]] = None, environment_variables: Optional[Dict[str, str]] = None, replica_count: int = 1, machine_type: str = 'n1-standard-4', accelerator_type: str = 'ACCELERATOR_TYPE_UNSPECIFIED', accelerator_count: int = 0, boot_disk_type: str = 'pd-ssd', boot_disk_size_gb: int = 100, reduction_server_replica_count: int = 0, reduction_server_machine_type: Optional[str] = None, reduction_server_container_uri: Optional[str] = None, training_fraction_split: Optional[float] = None, validation_fraction_split: Optional[float] = None, test_fraction_split: Optional[float] = None, training_filter_split: Optional[str] = None, validation_filter_split: Optional[str] = None, test_filter_split: Optional[str] = None, predefined_split_column_name: Optional[str] = None, timestamp_split_column_name: Optional[str] = None, timeout: Optional[int] = None, restart_job_on_worker_restart: bool = False, enable_web_access: bool = False, enable_dashboard_access: bool = False, tensorboard: Optional[str] = None, sync=True, create_request_timeout: Optional[float] = None, disable_retries: bool = False, persistent_resource_id: Optional[str] = None, tpu_topology: Optional[str] = None)

Submits the custom training job without blocking until completion.

Distributed Training Support: If replica count = 1 then one chief replica will be provisioned. If replica_count > 1 the remainder will be provisioned as a worker replica pool. ie: replica_count = 10 will result in 1 chief and 9 workers All replicas have same machine_type, accelerator_type, and accelerator_count

If training on a Vertex AI dataset, you can use one of the following split configurations:

Data fraction splits:
Any of `training_fraction_split`, `validation_fraction_split` and
`test_fraction_split` may optionally be provided, they must sum to up to 1. If
the provided ones sum to less than 1, the remainder is assigned to sets as
decided by Vertex AI. If none of the fractions are set, by default roughly 80%
of data will be used for training, 10% for validation, and 10% for test.

Data filter splits:
Assigns input data to training, validation, and test sets
based on the given filters, data pieces not matched by any
filter are ignored. Currently only supported for Datasets
containing DataItems.
If any of the filters in this message are to match nothing, then
they can be set as ‘-’ (the minus sign).
If using filter splits, all of `training_filter_split`, `validation_filter_split` and
`test_filter_split` must be provided.
Supported only for unstructured Datasets.

Predefined splits:
Assigns input data to training, validation, and test sets based on the value of a provided key.
If using predefined splits, `predefined_split_column_name` must be provided.
Supported only for tabular Datasets.

Timestamp splits:
Assigns input data to training, validation, and test sets
based on a provided timestamps. The youngest data pieces are
assigned to training set, next to validation set, and the oldest
to the test set.
Supported only for tabular Datasets.

  • Parameters

    • dataset (Union[datasets.ImageDataset,datasets.TabularDataset,datasets.TextDataset,datasets.VideoDataset]) – Vertex AI to fit this training against. Custom training script should retrieve datasets through passed in environment variables uris:

      os.environ[“AIP_TRAINING_DATA_URI”] os.environ[“AIP_VALIDATION_DATA_URI”] os.environ[“AIP_TEST_DATA_URI”]

      Additionally the dataset format is passed in as:

      os.environ[“AIP_DATA_FORMAT”]

    • annotation_schema_uri (str) – Google Cloud Storage URI points to a YAML file describing annotation schema. The schema is defined as an OpenAPI 3.0.2 Schema Object The schema files that can be used here are found in gs://google-cloud-aiplatform/schema/dataset/annotation/, note that the chosen schema must be consistent with metadata of the Dataset specified by dataset_id.

      Only Annotations that both match this schema and belong to DataItems not ignored by the split method are used in respectively training, validation or test role, depending on the role of the DataItem they are on.

      When used in conjunction with annotations_filter, the Annotations used for training are filtered by both annotations_filter and annotation_schema_uri.

    • model_display_name (str) – If the script produces a managed Vertex AI Model. The display name of the Model. The name can be up to 128 characters long and can be consist of any UTF-8 characters.

      If not provided upon creation, the job’s display_name is used.

    • model_labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The labels with user-defined metadata to organize your Models. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information and examples of labels.

    • model_id (str) – Optional. The ID to use for the Model produced by this job, which will become the final component of the model resource name. This value may be up to 63 characters, and valid characters are [a-z0-9_-]. The first character cannot be a number or hyphen.

    • parent_model (str) – Optional. The resource name or model ID of an existing model. The new model uploaded by this job will be a version of parent_model.

      Only set this field when training a new version of an existing model.

    • is_default_version (bool) – Optional. When set to True, the newly uploaded model version will automatically have alias “default” included. Subsequent uses of the model produced by this job without a version specified will use this “default” version.

      When set to False, the “default” alias will not be moved. Actions targeting the model version produced by this job will need to specifically reference this version by ID or alias.

      New model uploads, i.e. version 1, will always be “default” aliased.

    • model_version_aliases (Sequence[str]) – Optional. User provided version aliases so that the model version uploaded by this job can be referenced via alias instead of auto-generated version ID. A default version alias will be created for the first version of the model.

      The format is [a-z][a-zA-Z0-9-]{0,126}[a-z0-9]

    • model_version_description (str) – Optional. The description of the model version being uploaded by this job.

    • base_output_dir (str) – GCS output directory of job. If not provided a timestamped directory in the staging directory will be used.

      Vertex AI sets the following environment variables when it runs your training code:

      • AIP_MODEL_DIR: a Cloud Storage URI of a directory intended for saving model artifacts, i.e. <base_output_dir>/model/

      • AIP_CHECKPOINT_DIR: a Cloud Storage URI of a directory intended for saving checkpoints, i.e. <base_output_dir>/checkpoints/

      • AIP_TENSORBOARD_LOG_DIR: a Cloud Storage URI of a directory intended for saving TensorBoard logs, i.e. <base_output_dir>/logs/

    • service_account (str) – Specifies the service account for workload run-as account. Users submitting jobs must have act-as permission on this run-as account.

    • network (str) – The full name of the Compute Engine network to which the job should be peered. For example, projects/12345/global/networks/myVPC. Private services access must already be configured for the network. If left unspecified, the network set in aiplatform.init will be used. Otherwise, the job is not peered with any network.

    • bigquery_destination (str) – Provide this field if dataset is a BigQuery dataset. The BigQuery project location where the training data is to be written to. In the given project a new dataset is created with name dataset_<dataset-id>_<annotation-type>_<timestamp-of-training-call> where timestamp is in YYYY_MM_DDThh_mm_ss_sssZ format. All training input data will be written into that dataset. In the dataset three tables will be created, training, validation and test.

      • AIP_DATA_FORMAT = “bigquery”.

      • AIP_TRAINING_DATA_URI =”bigquery_destination.dataset_*.training”

      • AIP_VALIDATION_DATA_URI = “bigquery_destination.dataset_*.validation”

      • AIP_TEST_DATA_URI = “bigquery_destination.dataset_*.test”

    • args (List[Unions[str, *[int](https://python.readthedocs.io/en/latest/library/functions.html#int), [float](https://python.readthedocs.io/en/latest/library/functions.html#float)]*]) – Command line arguments to be passed to the Python script.

    • environment_variables (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Environment variables to be passed to the container. Should be a dictionary where keys are environment variable names and values are environment variable values for those names. At most 10 environment variables can be specified. The Name of the environment variable must be unique.

      environment_variables = {

      ‘MY_KEY’: ‘MY_VALUE’

      }

    • replica_count (int) – The number of worker replicas. If replica count = 1 then one chief replica will be provisioned. If replica_count > 1 the remainder will be provisioned as a worker replica pool.

    • machine_type (str) – The type of machine to use for training.

    • accelerator_type (str) – Hardware accelerator type. One of ACCELERATOR_TYPE_UNSPECIFIED, NVIDIA_TESLA_K80, NVIDIA_TESLA_P100, NVIDIA_TESLA_V100, NVIDIA_TESLA_P4, NVIDIA_TESLA_T4

    • accelerator_count (int) – The number of accelerators to attach to a worker replica.

    • boot_disk_type (str) – Type of the boot disk, default is pd-ssd. Valid values: pd-ssd (Persistent Disk Solid State Drive) or pd-standard (Persistent Disk Hard Disk Drive).

    • boot_disk_size_gb (int) – Size in GB of the boot disk, default is 100GB. boot disk size must be within the range of [100, 64000].

    • reduction_server_replica_count (int) – The number of reduction server replicas, default is 0.

    • reduction_server_machine_type (str) – Optional. The type of machine to use for reduction server.

    • reduction_server_container_uri (str) – Optional. The Uri of the reduction server container image. See details: https://cloud.google.com/vertex-ai/docs/training/distributed-training#reduce_training_time_with_reduction_server

    • training_fraction_split (float) – Optional. The fraction of the input data that is to be used to train the Model. This is ignored if Dataset is not provided.

    • validation_fraction_split (float) – Optional. The fraction of the input data that is to be used to validate the Model. This is ignored if Dataset is not provided.

    • test_fraction_split (float) – Optional. The fraction of the input data that is to be used to evaluate the Model. This is ignored if Dataset is not provided.

    • training_filter_split (str) – Optional. A filter on DataItems of the Dataset. DataItems that match this filter are used to train the Model. A filter with same syntax as the one used in DatasetService.ListDataItems may be used. If a single DataItem is matched by more than one of the FilterSplit filters, then it is assigned to the first set that applies to it in the training, validation, test order. This is ignored if Dataset is not provided.

    • validation_filter_split (str) – Optional. A filter on DataItems of the Dataset. DataItems that match this filter are used to validate the Model. A filter with same syntax as the one used in DatasetService.ListDataItems may be used. If a single DataItem is matched by more than one of the FilterSplit filters, then it is assigned to the first set that applies to it in the training, validation, test order. This is ignored if Dataset is not provided.

    • test_filter_split (str) – Optional. A filter on DataItems of the Dataset. DataItems that match this filter are used to test the Model. A filter with same syntax as the one used in DatasetService.ListDataItems may be used. If a single DataItem is matched by more than one of the FilterSplit filters, then it is assigned to the first set that applies to it in the training, validation, test order. This is ignored if Dataset is not provided.

    • predefined_split_column_name (str) – Optional. The key is a name of one of the Dataset’s data columns. The value of the key (either the label’s value or value in the column) must be one of {training, validation, test}, and it defines to which set the given piece of data is assigned. If for a piece of data the key is not present or has an invalid value, that piece is ignored by the pipeline.

      Supported only for tabular and time series Datasets.

    • timestamp_split_column_name (str) – Optional. The key is a name of one of the Dataset’s data columns. The value of the key values of the key (the values in the column) must be in RFC 3339 date-time format, where time-offset = “Z” (e.g. 1985-04-12T23:20:50.52Z). If for a piece of data the key is not present or has an invalid value, that piece is ignored by the pipeline.

      Supported only for tabular and time series Datasets.

    • timeout (int) – The maximum job running time in seconds. The default is 7 days.

    • restart_job_on_worker_restart (bool) – Restarts the entire CustomJob if a worker gets restarted. This feature can be used by distributed training jobs that are not resilient to workers leaving and joining a job.

    • enable_web_access (bool) – Whether you want Vertex AI to enable interactive shell access to training containers. https://cloud.google.com/vertex-ai/docs/training/monitor-debug-interactive-shell

    • enable_dashboard_access (bool) – Whether you want Vertex AI to enable access to the customized dashboard to training containers.

    • tensorboard (str) – Optional. The name of a Vertex AI [Tensorboard][google.cloud.aiplatform.v1beta1.Tensorboard] resource to which this CustomJob will upload Tensorboard logs. Format: projects/{project}/locations/{location}/tensorboards/{tensorboard}

      The training script should write Tensorboard to following Vertex AI environment variable:

      AIP_TENSORBOARD_LOG_DIR

      service_account is required with provided tensorboard. For more information on configuring your service account please visit: https://cloud.google.com/vertex-ai/docs/experiments/tensorboard-training

    • sync (bool) – Whether to execute this method synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

    • create_request_timeout (float) – Optional. The timeout for the create request in seconds.

    • disable_retries (bool) – Indicates if the job should retry for internal errors after the job starts running. If True, overrides restart_job_on_worker_restart to False.

    • persistent_resource_id (str) – Optional. The ID of the PersistentResource in the same Project and Location. If this is specified, the job will be run on existing machines held by the PersistentResource instead of on-demand short-live machines. The network, CMEK, and node pool configs on the job should be consistent with those on the PersistentResource, otherwise, the job will be rejected.

    • tpu_topology (str) – Optional. Specifies the tpu topology to be used for TPU training job. This field is required for TPU v5 versions. For details on the TPU topology, refer to https://cloud.google.com/tpu/docs/v5e#tpu-v5e-config. The topology must be a supported value for the TPU machine type.

  • Returns

    The trained Vertex AI Model resource or None if training did not

      produce a Vertex AI Model.

  • Return type

    model

  • Raises

    RuntimeError – If Training job has already been run, staging_bucket has not been set, or model_display_name was provided but required arguments were not provided in constructor.

to_dict()

Returns the resource proto as a dictionary.

property update_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was last updated.

wait()

Helper method that blocks until all futures are complete.

wait_for_resource_creation()

Waits until resource has been created.

property web_access_uris(: Dict[str, str )

Get the web access uris of the backing custom job.

  • Returns

    Web access uris of the backing custom job.

  • Return type

    (Dict[str, str])

class google.cloud.aiplatform.CustomJob(display_name: str, worker_pool_specs: Union[List[Dict], List[google.cloud.aiplatform_v1.types.custom_job.WorkerPoolSpec]], base_output_dir: Optional[str] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None, labels: Optional[Dict[str, str]] = None, encryption_spec_key_name: Optional[str] = None, staging_bucket: Optional[str] = None, persistent_resource_id: Optional[str] = None)

Bases: google.cloud.aiplatform.jobs._RunnableJob, google.cloud.aiplatform.base.PreviewMixin

Vertex AI Custom Job.

Constructs a Custom Job with Worker Pool Specs.

``

` Example usage: worker_pool_specs = [

{

“machine_spec”: {

    “machine_type”: “n1-standard-4”,
    “accelerator_type”: “NVIDIA_TESLA_K80”,
    “accelerator_count”: 1,

},
“replica_count”: 1,
“container_spec”: {

> “image_uri”: container_image_uri,
> “command”: [],
> “args”: [],

},

}

]

my_job = aiplatform.CustomJob(

display_name=’my_job’,
worker_pool_specs=worker_pool_specs,
labels={‘my_key’: ‘my_value’},

)

my_job.run()

``

`

For more information on configuring worker pool specs please visit: https://cloud.google.com/ai-platform-unified/docs/training/create-custom-job

  • Parameters

    • display_name (str) – Required. The user-defined name of the HyperparameterTuningJob. The name can be up to 128 characters long and can be consist of any UTF-8 characters.

    • worker_pool_specs (Union[List[Dict], **List[aiplatform.gapic.WorkerPoolSpec]]) – Required. The spec of the worker pools including machine type and Docker image. Can provided as a list of dictionaries or list of WorkerPoolSpec proto messages.

    • base_output_dir (str) – Optional. GCS output directory of job. If not provided a timestamped directory in the staging directory will be used.

    • project (str) – Optional.Project to run the custom job in. Overrides project set in aiplatform.init.

    • location (str) – Optional.Location to run the custom job in. Overrides location set in aiplatform.init.

    • credentials (auth_credentials.Credentials) – Optional.Custom credentials to use to run call custom job service. Overrides credentials set in aiplatform.init.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The labels with user-defined metadata to organize CustomJobs. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information and examples of labels.

    • encryption_spec_key_name (str) – Optional.Customer-managed encryption key name for a CustomJob. If this is set, then all resources created by the CustomJob will be encrypted with the provided encryption key.

    • staging_bucket (str) – Optional. Bucket for produced custom job artifacts. Overrides staging_bucket set in aiplatform.init.

    • persistent_resource_id (str) – Optional. The ID of the PersistentResource in the same Project and Location. If this is specified, the job will be run on existing machines held by the PersistentResource instead of on-demand short-live machines. The network and CMEK configs on the job should be consistent with those on the PersistentResource, otherwise, the job will be rejected.

  • Raises

    RuntimeError – If staging bucket was not set using aiplatform.init and a staging bucket was not passed in.

cancel()

Cancels this Job.

Success of cancellation is not guaranteed. Use Job.state property to verify if cancellation was successful.

client_class()

alias of google.cloud.aiplatform.utils.JobClientWithOverride

property create_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was created.

delete(sync: bool = True)

Deletes this Vertex AI resource. WARNING: This deletion is permanent.

  • Parameters

    sync (bool) – Whether to execute this deletion synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

property display_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Display name of this resource.

done()

Method indicating whether a job has completed.

  • Returns

    True if the job has completed.

property encryption_spec(: Optional[[google.cloud.aiplatform_v1.types.encryption_spec.EncryptionSpec](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.EncryptionSpec)_ )

Customer-managed encryption key options for this Vertex AI resource.

If this is set, then all resources created by this Vertex AI resource will be encrypted with the provided encryption key.

property end_time(: Optional[datetime.datetime )

Time when the Job resource entered the JOB_STATE_SUCCEEDED, JOB_STATE_FAILED, or JOB_STATE_CANCELLED state.

property error(: Optional[google.rpc.status_pb2.Status )

Detailed error info for this Job resource. Only populated when the Job’s state is JOB_STATE_FAILED or JOB_STATE_CANCELLED.

classmethod from_local_script(display_name: str, script_path: str, container_uri: str, enable_autolog: bool = False, args: Optional[Sequence[str]] = None, requirements: Optional[Sequence[str]] = None, environment_variables: Optional[Dict[str, str]] = None, replica_count: int = 1, machine_type: str = 'n1-standard-4', accelerator_type: str = 'ACCELERATOR_TYPE_UNSPECIFIED', accelerator_count: int = 0, boot_disk_type: str = 'pd-ssd', boot_disk_size_gb: int = 100, reduction_server_replica_count: int = 0, reduction_server_machine_type: Optional[str] = None, reduction_server_container_uri: Optional[str] = None, base_output_dir: Optional[str] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None, labels: Optional[Dict[str, str]] = None, encryption_spec_key_name: Optional[str] = None, staging_bucket: Optional[str] = None, persistent_resource_id: Optional[str] = None, tpu_topology: Optional[str] = None)

Configures a custom job from a local script.

Example usage:

``

` job = aiplatform.CustomJob.from_local_script(

display_name=”my-custom-job”, script_path=”training_script.py”, container_uri=”gcr.io/cloud-aiplatform/training/tf-cpu.2-2:latest”, requirements=[“gcsfs==0.7.1”], replica_count=1, args=[’–dataset’, ‘gs://my-bucket/my-dataset’, ‘–model_output_uri’, ‘gs://my-bucket/model’] labels={‘my_key’: ‘my_value’},

)

job.run()

``

`

  • Parameters

    • display_name (str) – Required. The user-defined name of this CustomJob.

    • script_path (str) – Required. Local path to training script.

    • container_uri (str) – Required. Uri of the training container image to use for custom job. Support images in Artifact Registry, Container Registry, or Docker Hub. Vertex AI provides a wide range of executor images with pre-installed packages to meet users’ various use cases. See the list of pre-built containers for training https://cloud.google.com/vertex-ai/docs/training/pre-built-containers. If not using image from this list, please make sure python3 and pip3 are installed in your container.

    • enable_autolog (bool) – Optional. If True, the Vertex Experiments autologging feature will be enabled in the CustomJob. Note that this will wrap your training script with some autologging-related code.

    • args (Optional[Sequence[str]]) – Optional. Command line arguments to be passed to the Python task.

    • requirements (Sequence[str]) – Optional. List of python packages dependencies of script.

    • environment_variables (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. Environment variables to be passed to the container. Should be a dictionary where keys are environment variable names and values are environment variable values for those names. At most 10 environment variables can be specified. The Name of the environment variable must be unique.

      environment_variables = {

      ‘MY_KEY’: ‘MY_VALUE’

      }

    • replica_count (int) – Optional. The number of worker replicas. If replica count = 1 then one chief replica will be provisioned. If replica_count > 1 the remainder will be provisioned as a worker replica pool.

    • machine_type (str) – Optional. The type of machine to use for training.

    • accelerator_type (str) – Optional. Hardware accelerator type. One of ACCELERATOR_TYPE_UNSPECIFIED, NVIDIA_TESLA_K80, NVIDIA_TESLA_P100, NVIDIA_TESLA_V100, NVIDIA_TESLA_P4, NVIDIA_TESLA_T4

    • accelerator_count (int) – Optional. The number of accelerators to attach to a worker replica.

    • boot_disk_type (str) – Optional. Type of the boot disk, default is pd-ssd. Valid values: pd-ssd (Persistent Disk Solid State Drive) or pd-standard (Persistent Disk Hard Disk Drive).

    • boot_disk_size_gb (int) – Optional. Size in GB of the boot disk, default is 100GB. boot disk size must be within the range of [100, 64000].

    • reduction_server_replica_count (int) – The number of reduction server replicas, default is 0.

    • reduction_server_machine_type (str) – Optional. The type of machine to use for reduction server.

    • reduction_server_container_uri (str) – Optional. The Uri of the reduction server container image. See details: https://cloud.google.com/vertex-ai/docs/training/distributed-training#reduce_training_time_with_reduction_server

    • base_output_dir (str) – Optional. GCS output directory of job. If not provided a timestamped directory in the staging directory will be used.

    • project (str) – Optional. Project to run the custom job in. Overrides project set in aiplatform.init.

    • location (str) – Optional. Location to run the custom job in. Overrides location set in aiplatform.init.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to run call custom job service. Overrides credentials set in aiplatform.init.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The labels with user-defined metadata to organize CustomJobs. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information and examples of labels.

    • encryption_spec_key_name (str) – Optional. Customer-managed encryption key name for a CustomJob. If this is set, then all resources created by the CustomJob will be encrypted with the provided encryption key.

    • staging_bucket (str) – Optional. Bucket for produced custom job artifacts. Overrides staging_bucket set in aiplatform.init.

    • persistent_resource_id (str) – Optional. The ID of the PersistentResource in the same Project and Location. If this is specified, the job will be run on existing machines held by the PersistentResource instead of on-demand short-live machines. The network, CMEK, and node pool configs on the job should be consistent with those on the PersistentResource, otherwise, the job will be rejected.

    • tpu_topology (str) – Optional. Specifies the tpu topology to be used for TPU training job. This field is required for TPU v5 versions. For details on the TPU topology, refer to https://cloud.google.com/tpu/docs/v5e#tpu-v5e-config. The topology must be a supported value for the TPU machine type.

  • Raises

    RuntimeError – If staging bucket was not set using aiplatform.init and a staging bucket was not passed in.

property gca_resource(: [proto.message.Message](https://proto-plus-python.readthedocs.io/en/latest/reference/message.html#proto.message.Message )

The underlying resource proto representation.

classmethod get(resource_name: str, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Get a Vertex AI Job for the given resource_name.

  • Parameters

    • resource_name (str) – Required. A fully-qualified resource name or ID.

    • project (str) – Optional. project to retrieve dataset from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. location to retrieve dataset from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Custom credentials to use to upload this model. Overrides credentials set in aiplatform.init.

  • Returns

    A Vertex AI Job.

property labels(: Dict[str, str )

User-defined labels containing metadata about this resource.

Read more about labels at https://goo.gl/xmQnxf

classmethod list(filter: Optional[str] = None, order_by: Optional[str] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

List all instances of this Job Resource.

Example Usage:

aiplatform.BatchPredictionJobs.list(

filter=’state=”JOB_STATE_SUCCEEDED” AND display_name=”my_job”’,

)

  • Parameters

    • filter (str) – Optional. An expression for filtering the results of the request. For field names both snake_case and camelCase are supported.

    • order_by (str) – Optional. A comma-separated list of fields to order by, sorted in ascending order. Use “desc” after a field name for descending. Supported fields: display_name, create_time, update_time

    • project (str) – Optional. Project to retrieve list from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to retrieve list from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to retrieve list. Overrides credentials set in aiplatform.init.

  • Returns

    List[VertexAiResourceNoun] - A list of Job resource objects.

property name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Name of this resource.

property network(: Optional[str )

The full name of the Google Compute Engine network to which this CustomJob should be peered.

Takes the format projects/{project}/global/networks/{network}. Where {project} is a project number, as in 12345, and {network} is a network name.

Private services access must already be configured for the network. If left unspecified, the CustomJob is not peered with any network.

property preview(: google.cloud.aiplatform.base.PreviewClas )

Exposes features available in preview for this class.

property resource_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Full qualified resource name.

run(service_account: Optional[str] = None, network: Optional[str] = None, timeout: Optional[int] = None, restart_job_on_worker_restart: bool = False, enable_web_access: bool = False, experiment: Optional[Union[google.cloud.aiplatform.metadata.experiment_resources.Experiment, str]] = None, experiment_run: Optional[Union[google.cloud.aiplatform.metadata.experiment_run_resource.ExperimentRun, str]] = None, tensorboard: Optional[str] = None, sync: bool = True, create_request_timeout: Optional[float] = None, disable_retries: bool = False, persistent_resource_id: Optional[str] = None)

Run this configured CustomJob.

  • Parameters

    • service_account (str) – Optional. Specifies the service account for workload run-as account. Users submitting jobs must have act-as permission on this run-as account.

    • network (str) – Optional. The full name of the Compute Engine network to which the job should be peered. For example, projects/12345/global/networks/myVPC. Private services access must already be configured for the network. If left unspecified, the network set in aiplatform.init will be used. Otherwise, the job is not peered with any network.

    • timeout (int) – The maximum job running time in seconds. The default is 7 days.

    • restart_job_on_worker_restart (bool) – Restarts the entire CustomJob if a worker gets restarted. This feature can be used by distributed training jobs that are not resilient to workers leaving and joining a job.

    • enable_web_access (bool) – Whether you want Vertex AI to enable interactive shell access to training containers. https://cloud.google.com/vertex-ai/docs/training/monitor-debug-interactive-shell

    • experiment (Union[aiplatform.Experiment, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The instance or name of an Experiment resource to which this CustomJob will upload training parameters and metrics.

      service_account is required with provided experiment. For more information on configuring your service account please visit: https://cloud.google.com/vertex-ai/docs/experiments/tensorboard-training

    • experiment_run (Union[aiplatform.ExperimentRun, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The instance or name of an ExperimentRun resource to which this CustomJob will upload training parameters and metrics. This arg can only be set when experiment is set. If ‘experiment’ is set but ‘experiment_run` is not, an ExperimentRun resource will still be auto-generated.

    • tensorboard (str) – Optional. The name of a Vertex AI [Tensorboard][google.cloud.aiplatform.v1beta1.Tensorboard] resource to which this CustomJob will upload Tensorboard logs. Format: projects/{project}/locations/{location}/tensorboards/{tensorboard}

      The training script should write Tensorboard to following Vertex AI environment variable:

      AIP_TENSORBOARD_LOG_DIR

      service_account is required with provided tensorboard. For more information on configuring your service account please visit: https://cloud.google.com/vertex-ai/docs/experiments/tensorboard-training

    • sync (bool) – Whether to execute this method synchronously. If False, this method will unblock and it will be executed in a concurrent Future.

    • create_request_timeout (float) – Optional. The timeout for the create request in seconds.

    • disable_retries (bool) – Indicates if the job should retry for internal errors after the job starts running. If True, overrides restart_job_on_worker_restart to False.

    • persistent_resource_id (str) – Optional. The ID of the PersistentResource in the same Project and Location. If this is specified, the job will be run on existing machines held by the PersistentResource instead of on-demand short-live machines. The network, CMEK, and node pool configs on the job should be consistent with those on the PersistentResource, otherwise, the job will be rejected.

property start_time(: Optional[datetime.datetime )

Time when the Job resource entered the JOB_STATE_RUNNING for the first time.

property state(: [google.cloud.aiplatform_v1.types.job_state.JobState](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.JobState_ )

Fetch Job again and return the current JobState.

  • Returns

    Enum that describes the state of a Vertex AI job.

  • Return type

    state (job_state.JobState)

submit(*, service_account: Optional[str] = None, network: Optional[str] = None, timeout: Optional[int] = None, restart_job_on_worker_restart: bool = False, enable_web_access: bool = False, experiment: Optional[Union[google.cloud.aiplatform.metadata.experiment_resources.Experiment, str]] = None, experiment_run: Optional[Union[google.cloud.aiplatform.metadata.experiment_run_resource.ExperimentRun, str]] = None, tensorboard: Optional[str] = None, create_request_timeout: Optional[float] = None, disable_retries: bool = False, persistent_resource_id: Optional[str] = None)

Submit the configured CustomJob.

  • Parameters

    • service_account (str) – Optional. Specifies the service account for workload run-as account. Users submitting jobs must have act-as permission on this run-as account.

    • network (str) – Optional. The full name of the Compute Engine network to which the job should be peered. For example, projects/12345/global/networks/myVPC. Private services access must already be configured for the network.

    • timeout (int) – The maximum job running time in seconds. The default is 7 days.

    • restart_job_on_worker_restart (bool) – Restarts the entire CustomJob if a worker gets restarted. This feature can be used by distributed training jobs that are not resilient to workers leaving and joining a job.

    • enable_web_access (bool) – Whether you want Vertex AI to enable interactive shell access to training containers. https://cloud.google.com/vertex-ai/docs/training/monitor-debug-interactive-shell

    • experiment (Union[aiplatform.Experiment, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The instance or name of an Experiment resource to which this CustomJob will upload training parameters and metrics.

      service_account is required with provided experiment. For more information on configuring your service account please visit: https://cloud.google.com/vertex-ai/docs/experiments/tensorboard-training

    • experiment_run (Union[aiplatform.ExperimentRun, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The instance or name of an ExperimentRun resource to which this CustomJob will upload training parameters and metrics. This arg can only be set when experiment is set. If ‘experiment’ is set but ‘experiment_run` is not, an ExperimentRun resource will still be auto-generated.

    • tensorboard (str) – Optional. The name of a Vertex AI [Tensorboard][google.cloud.aiplatform.v1beta1.Tensorboard] resource to which this CustomJob will upload Tensorboard logs. Format: projects/{project}/locations/{location}/tensorboards/{tensorboard}

      The training script should write Tensorboard to following Vertex AI environment variable:

      AIP_TENSORBOARD_LOG_DIR

      service_account is required with provided tensorboard. For more information on configuring your service account please visit: https://cloud.google.com/vertex-ai/docs/experiments/tensorboard-training

    • create_request_timeout (float) – Optional. The timeout for the create request in seconds.

    • disable_retries (bool) – Indicates if the job should retry for internal errors after the job starts running. If True, overrides restart_job_on_worker_restart to False.

    • persistent_resource_id (str) – Optional. The ID of the PersistentResource in the same Project and Location. If this is specified, the job will be run on existing machines held by the PersistentResource instead of on-demand short-live machines. The network, CMEK, and node pool configs on the job should be consistent with those on the PersistentResource, otherwise, the job will be rejected.

  • Raises

    ValueError – If both experiment and tensorboard are specified or if enable_autolog is True in CustomJob.from_local_script but experiment is not specified or the specified experiment doesn’t have a backing tensorboard.

to_dict()

Returns the resource proto as a dictionary.

property update_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was last updated.

wait()

Helper method that blocks until all futures are complete.

wait_for_completion()

Waits for job to complete.

wait_for_resource_creation()

Waits until resource has been created.

property web_access_uris(: Dict[str, Union[str, Dict[str, str]] )

Fetch the runnable job again and return the latest web access uris.

  • Returns

    Web access uris of the runnable job.

  • Return type

    (Dict[str, Union[str, Dict[str, str]]])

class google.cloud.aiplatform.CustomPythonPackageTrainingJob(display_name: str, python_package_gcs_uri: Union[str, List[str]], python_module_name: str, container_uri: str, model_serving_container_image_uri: Optional[str] = None, model_serving_container_predict_route: Optional[str] = None, model_serving_container_health_route: Optional[str] = None, model_serving_container_command: Optional[Sequence[str]] = None, model_serving_container_args: Optional[Sequence[str]] = None, model_serving_container_environment_variables: Optional[Dict[str, str]] = None, model_serving_container_ports: Optional[Sequence[int]] = None, model_description: Optional[str] = None, model_instance_schema_uri: Optional[str] = None, model_parameters_schema_uri: Optional[str] = None, model_prediction_schema_uri: Optional[str] = None, explanation_metadata: Optional[google.cloud.aiplatform_v1.types.explanation_metadata.ExplanationMetadata] = None, explanation_parameters: Optional[google.cloud.aiplatform_v1.types.explanation.ExplanationParameters] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None, labels: Optional[Dict[str, str]] = None, training_encryption_spec_key_name: Optional[str] = None, model_encryption_spec_key_name: Optional[str] = None, staging_bucket: Optional[str] = None)

Bases: google.cloud.aiplatform.training_jobs._CustomTrainingJob

Class to launch a Custom Training Job in Vertex AI using a Python Package.

Takes a training implementation as a python package and executes that package in Cloud Vertex AI Training.

Constructs a Custom Training Job from a Python Package.

job = aiplatform.CustomPythonPackageTrainingJob(

display_name=’test-train’,
python_package_gcs_uri=’gs://my-bucket/my-python-package.tar.gz’,
python_module_name=’my-training-python-package.task’,
container_uri=’gcr.io/cloud-aiplatform/training/tf-cpu.2-2:latest’,
model_serving_container_image_uri=’gcr.io/my-trainer/serving:1’,
model_serving_container_predict_route=’predict’,
model_serving_container_health_route=’metadata,
labels={‘key’: ‘value’},

)

Usage with Dataset:

ds = aiplatform.TabularDataset(

‘projects/my-project/locations/us-central1/datasets/12345’

)

job.run(

ds,
replica_count=1,
model_display_name=’my-trained-model’,
model_labels={‘key’: ‘value’},

)

Usage without Dataset:

job.run(

replica_count=1,
model_display_name=’my-trained-model’,
model_labels={‘key’: ‘value’},

)

To ensure your model gets saved in Vertex AI, write your saved model to os.environ[“AIP_MODEL_DIR”] in your provided training script.

  • Parameters

    • display_name (str) – Required. The user-defined name of this TrainingPipeline.

    • python_package_gcs_uri (Union[str, **List[str]]) – Required. GCS location of the training python package. Could be a string for single package or a list of string for multiple packages.

    • python_module_name (str) – Required. The module name of the training python package.

    • container_uri (str) – Required. Uri of the training container image in the GCR.

    • model_serving_container_image_uri (str) – Optional. If the training produces a managed Vertex AI Model, the URI of the model serving container suitable for serving the model produced by the training script.

    • model_serving_container_predict_route (str) – Optional. If the training produces a managed Vertex AI Model, an HTTP path to send prediction requests to the container, and which must be supported by it. If not specified a default HTTP path will be used by Vertex AI.

    • model_serving_container_health_route (str) – Optional. If the training produces a managed Vertex AI Model, an HTTP path to send health check requests to the container, and which must be supported by it. If not specified a standard HTTP path will be used by AI Platform.

    • model_serving_container_command (Sequence[str]) – Optional. The command with which the container is run. Not executed within a shell. The Docker image’s ENTRYPOINT is used if this is not provided. Variable references $(VAR_NAME) are expanded using the container’s environment. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not.

    • model_serving_container_args (Sequence[str]) – Optional. The arguments to the command. The Docker image’s CMD is used if this is not provided. Variable references $(VAR_NAME) are expanded using the container’s environment. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not.

    • model_serving_container_environment_variables (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The environment variables that are to be present in the container. Should be a dictionary where keys are environment variable names and values are environment variable values for those names.

    • model_serving_container_ports (Sequence[int]) – Optional. Declaration of ports that are exposed by the container. This field is primarily informational, it gives Vertex AI information about the network connections the container uses. Listing or not a port here has no impact on whether the port is actually exposed, any port listening on the default “0.0.0.0” address inside a container will be accessible from the network.

    • model_description (str) – Optional. The description of the Model.

    • model_instance_schema_uri (str) – Optional. Points to a YAML file stored on Google Cloud Storage describing the format of a single instance, which are used in PredictRequest.instances, ExplainRequest.instances and BatchPredictionJob.input_config. The schema is defined as an OpenAPI 3.0.2 Schema Object. AutoML Models always have this field populated by AI Platform. Note: The URI given on output will be immutable and probably different, including the URI scheme, than the one given on input. The output URI will point to a location where the user only has a read access.

    • model_parameters_schema_uri (str) – Optional. Points to a YAML file stored on Google Cloud Storage describing the parameters of prediction and explanation via PredictRequest.parameters, ExplainRequest.parameters and BatchPredictionJob.model_parameters. The schema is defined as an OpenAPI 3.0.2 Schema Object. AutoML Models always have this field populated by AI Platform, if no parameters are supported it is set to an empty string. Note: The URI given on output will be immutable and probably different, including the URI scheme, than the one given on input. The output URI will point to a location where the user only has a read access.

    • model_prediction_schema_uri (str) – Optional. Points to a YAML file stored on Google Cloud Storage describing the format of a single prediction produced by this Model, which are returned via PredictResponse.predictions, ExplainResponse.explanations, and BatchPredictionJob.output_config. The schema is defined as an OpenAPI 3.0.2 Schema Object. AutoML Models always have this field populated by AI Platform. Note: The URI given on output will be immutable and probably different, including the URI scheme, than the one given on input. The output URI will point to a location where the user only has a read access.

    • explanation_metadata (explain.ExplanationMetadata) – Optional. Metadata describing the Model’s input and output for explanation. explanation_metadata is optional while explanation_parameters must be specified when used. For more details, see Ref docs http://tinyurl.com/1igh60kt

    • explanation_parameters (explain.ExplanationParameters) – Optional. Parameters to configure explaining for Model’s predictions. For more details, see Ref docs http://tinyurl.com/1an4zake

    • project (str) – Project to run training in. Overrides project set in aiplatform.init.

    • location (str) – Location to run training in. Overrides location set in aiplatform.init.

    • credentials (auth_credentials.Credentials) – Custom credentials to use to run call training service. Overrides credentials set in aiplatform.init.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The labels with user-defined metadata to organize TrainingPipelines. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information and examples of labels.

    • training_encryption_spec_key_name (Optional[str]) – Optional. The Cloud KMS resource identifier of the customer managed encryption key used to protect the training pipeline. Has the form: projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key. The key needs to be in the same region as where the compute resource is created.

      If set, this TrainingPipeline will be secured by this key.

      Note: Model trained by this TrainingPipeline is also secured by this key if model_to_upload is not set separately.

      Overrides encryption_spec_key_name set in aiplatform.init.

    • model_encryption_spec_key_name (Optional[str]) – Optional. The Cloud KMS resource identifier of the customer managed encryption key used to protect the model. Has the form: projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key. The key needs to be in the same region as where the compute resource is created.

      If set, the trained Model will be secured by this key.

      Overrides encryption_spec_key_name set in aiplatform.init.

    • staging_bucket (str) – Optional. Bucket used to stage source and training artifacts. Overrides staging_bucket set in aiplatform.init.

cancel()

Starts asynchronous cancellation on the TrainingJob. The server makes a best effort to cancel the job, but success is not guaranteed. On successful cancellation, the TrainingJob is not deleted; instead it becomes a job with state set to CANCELLED.

  • Raises

    RuntimeError – If this TrainingJob has not started running.

client_class()

alias of google.cloud.aiplatform.utils.PipelineClientWithOverride

property create_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was created.

delete(sync: bool = True)

Deletes this Vertex AI resource. WARNING: This deletion is permanent.

  • Parameters

    sync (bool) – Whether to execute this deletion synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

property display_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Display name of this resource.

done()

Method indicating whether a job has completed.

  • Returns

    True if the job has completed.

property encryption_spec(: Optional[[google.cloud.aiplatform_v1.types.encryption_spec.EncryptionSpec](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.EncryptionSpec)_ )

Customer-managed encryption key options for this Vertex AI resource.

If this is set, then all resources created by this Vertex AI resource will be encrypted with the provided encryption key.

property end_time(: Optional[datetime.datetime )

Time when the TrainingJob resource entered the PIPELINE_STATE_SUCCEEDED, PIPELINE_STATE_FAILED, PIPELINE_STATE_CANCELLED state.

property error(: Optional[google.rpc.status_pb2.Status )

Detailed error info for this TrainingJob resource. Only populated when the TrainingJob’s state is PIPELINE_STATE_FAILED or PIPELINE_STATE_CANCELLED.

property gca_resource(: [proto.message.Message](https://proto-plus-python.readthedocs.io/en/latest/reference/message.html#proto.message.Message )

The underlying resource proto representation.

classmethod get(resource_name: str, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Get Training Job for the given resource_name.

  • Parameters

    • resource_name (str) – Required. A fully-qualified resource name or ID.

    • project (str) – Optional project to retrieve training job from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional location to retrieve training job from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Custom credentials to use to upload this model. Overrides credentials set in aiplatform.init.

  • Raises

    ValueError – If the retrieved training job’s training task definition doesn’t match the custom training task definition.

  • Returns

    A Vertex AI Training Job

get_model(sync=True)

Vertex AI Model produced by this training, if one was produced.

  • Parameters

    sync (bool) – Whether to execute this method synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

  • Returns

    Vertex AI Model produced by this training

  • Return type

    model

  • Raises

    RuntimeError – If training failed or if a model was not produced by this training.

property has_failed(: [bool](https://python.readthedocs.io/en/latest/library/functions.html#bool )

Returns True if training has failed.

False otherwise.

property labels(: Dict[str, str )

User-defined labels containing metadata about this resource.

Read more about labels at https://goo.gl/xmQnxf

classmethod list(filter: Optional[str] = None, order_by: Optional[str] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

List all instances of this TrainingJob resource.

Example Usage:

aiplatform.CustomTrainingJob.list(

filter=’display_name=”experiment_a27”’,
order_by=’create_time desc’

)

  • Parameters

    • filter (str) – Optional. An expression for filtering the results of the request. For field names both snake_case and camelCase are supported.

    • order_by (str) – Optional. A comma-separated list of fields to order by, sorted in ascending order. Use “desc” after a field name for descending. Supported fields: display_name, create_time, update_time

    • project (str) – Optional. Project to retrieve list from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to retrieve list from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to retrieve list. Overrides credentials set in aiplatform.init.

  • Returns

    List[VertexAiResourceNoun] - A list of TrainingJob resource objects

property name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Name of this resource.

property network(: Optional[str )

The full name of the Google Compute Engine network to which this CustomTrainingJob should be peered.

Takes the format projects/{project}/global/networks/{network}. Where {project} is a project number, as in 12345, and {network} is a network name.

Private services access must already be configured for the network. If left unspecified, the CustomTrainingJob is not peered with any network.

property resource_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Full qualified resource name.

run(dataset: Optional[Union[google.cloud.aiplatform.datasets.image_dataset.ImageDataset, google.cloud.aiplatform.datasets.tabular_dataset.TabularDataset, google.cloud.aiplatform.datasets.text_dataset.TextDataset, google.cloud.aiplatform.datasets.video_dataset.VideoDataset]] = None, annotation_schema_uri: Optional[str] = None, model_display_name: Optional[str] = None, model_labels: Optional[Dict[str, str]] = None, model_id: Optional[str] = None, parent_model: Optional[str] = None, is_default_version: Optional[bool] = True, model_version_aliases: Optional[Sequence[str]] = None, model_version_description: Optional[str] = None, base_output_dir: Optional[str] = None, service_account: Optional[str] = None, network: Optional[str] = None, bigquery_destination: Optional[str] = None, args: Optional[List[Union[str, float, int]]] = None, environment_variables: Optional[Dict[str, str]] = None, replica_count: int = 1, machine_type: str = 'n1-standard-4', accelerator_type: str = 'ACCELERATOR_TYPE_UNSPECIFIED', accelerator_count: int = 0, boot_disk_type: str = 'pd-ssd', boot_disk_size_gb: int = 100, reduction_server_replica_count: int = 0, reduction_server_machine_type: Optional[str] = None, reduction_server_container_uri: Optional[str] = None, training_fraction_split: Optional[float] = None, validation_fraction_split: Optional[float] = None, test_fraction_split: Optional[float] = None, training_filter_split: Optional[str] = None, validation_filter_split: Optional[str] = None, test_filter_split: Optional[str] = None, predefined_split_column_name: Optional[str] = None, timestamp_split_column_name: Optional[str] = None, timeout: Optional[int] = None, restart_job_on_worker_restart: bool = False, enable_web_access: bool = False, enable_dashboard_access: bool = False, tensorboard: Optional[str] = None, sync=True, create_request_timeout: Optional[float] = None, disable_retries: bool = False, persistent_resource_id: Optional[str] = None, tpu_topology: Optional[str] = None)

Runs the custom training job.

Distributed Training Support: If replica count = 1 then one chief replica will be provisioned. If replica_count > 1 the remainder will be provisioned as a worker replica pool. ie: replica_count = 10 will result in 1 chief and 9 workers All replicas have same machine_type, accelerator_type, and accelerator_count

If training on a Vertex AI dataset, you can use one of the following split configurations:

Data fraction splits:
Any of `training_fraction_split`, `validation_fraction_split` and
`test_fraction_split` may optionally be provided, they must sum to up to 1. If
the provided ones sum to less than 1, the remainder is assigned to sets as
decided by Vertex AI. If none of the fractions are set, by default roughly 80%
of data will be used for training, 10% for validation, and 10% for test.

Data filter splits:
Assigns input data to training, validation, and test sets
based on the given filters, data pieces not matched by any
filter are ignored. Currently only supported for Datasets
containing DataItems.
If any of the filters in this message are to match nothing, then
they can be set as ‘-’ (the minus sign).
If using filter splits, all of `training_filter_split`, `validation_filter_split` and
`test_filter_split` must be provided.
Supported only for unstructured Datasets.

Predefined splits:
Assigns input data to training, validation, and test sets based on the value of a provided key.
If using predefined splits, `predefined_split_column_name` must be provided.
Supported only for tabular Datasets.

Timestamp splits:
Assigns input data to training, validation, and test sets
based on a provided timestamps. The youngest data pieces are
assigned to training set, next to validation set, and the oldest
to the test set.
Supported only for tabular Datasets.

  • Parameters

    • dataset (Union[datasets.ImageDataset,datasets.TabularDataset,datasets.TextDataset,datasets.VideoDataset,]) – Vertex AI to fit this training against. Custom training script should retrieve datasets through passed in environment variables uris:

      os.environ[“AIP_TRAINING_DATA_URI”] os.environ[“AIP_VALIDATION_DATA_URI”] os.environ[“AIP_TEST_DATA_URI”]

      Additionally the dataset format is passed in as:

      os.environ[“AIP_DATA_FORMAT”]

    • annotation_schema_uri (str) – Google Cloud Storage URI points to a YAML file describing annotation schema. The schema is defined as an OpenAPI 3.0.2 Schema Object The schema files that can be used here are found in gs://google-cloud-aiplatform/schema/dataset/annotation/, note that the chosen schema must be consistent with metadata of the Dataset specified by dataset_id.

      Only Annotations that both match this schema and belong to DataItems not ignored by the split method are used in respectively training, validation or test role, depending on the role of the DataItem they are on.

      When used in conjunction with annotations_filter, the Annotations used for training are filtered by both annotations_filter and annotation_schema_uri.

    • model_display_name (str) – If the script produces a managed Vertex AI Model. The display name of the Model. The name can be up to 128 characters long and can be consist of any UTF-8 characters.

      If not provided upon creation, the job’s display_name is used.

    • model_labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The labels with user-defined metadata to organize your Models. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information and examples of labels.

    • model_id (str) – Optional. The ID to use for the Model produced by this job, which will become the final component of the model resource name. This value may be up to 63 characters, and valid characters are [a-z0-9_-]. The first character cannot be a number or hyphen.

    • parent_model (str) – Optional. The resource name or model ID of an existing model. The new model uploaded by this job will be a version of parent_model.

      Only set this field when training a new version of an existing model.

    • is_default_version (bool) – Optional. When set to True, the newly uploaded model version will automatically have alias “default” included. Subsequent uses of the model produced by this job without a version specified will use this “default” version.

      When set to False, the “default” alias will not be moved. Actions targeting the model version produced by this job will need to specifically reference this version by ID or alias.

      New model uploads, i.e. version 1, will always be “default” aliased.

    • model_version_aliases (Sequence[str]) – Optional. User provided version aliases so that the model version uploaded by this job can be referenced via alias instead of auto-generated version ID. A default version alias will be created for the first version of the model.

      The format is [a-z][a-zA-Z0-9-]{0,126}[a-z0-9]

    • model_version_description (str) – Optional. The description of the model version being uploaded by this job.

    • base_output_dir (str) – GCS output directory of job. If not provided a timestamped directory in the staging directory will be used.

      Vertex AI sets the following environment variables when it runs your training code:

      • AIP_MODEL_DIR: a Cloud Storage URI of a directory intended for saving model artifacts, i.e. <base_output_dir>/model/

      • AIP_CHECKPOINT_DIR: a Cloud Storage URI of a directory intended for saving checkpoints, i.e. <base_output_dir>/checkpoints/

      • AIP_TENSORBOARD_LOG_DIR: a Cloud Storage URI of a directory intended for saving TensorBoard logs, i.e. <base_output_dir>/logs/

    • service_account (str) – Specifies the service account for workload run-as account. Users submitting jobs must have act-as permission on this run-as account. If not specified, uses the service account set in aiplatform.init.

    • network (str) – The full name of the Compute Engine network to which the job should be peered. For example, projects/12345/global/networks/myVPC. Private services access must already be configured for the network. If left unspecified, the network set in aiplatform.init will be used. Otherwise, the job is not peered with any network.

    • bigquery_destination (str) – Provide this field if dataset is a BigQuery dataset. The BigQuery project location where the training data is to be written to. In the given project a new dataset is created with name dataset_<dataset-id>_<annotation-type>_<timestamp-of-training-call> where timestamp is in YYYY_MM_DDThh_mm_ss_sssZ format. All training input data will be written into that dataset. In the dataset three tables will be created, training, validation and test.

      • AIP_DATA_FORMAT = “bigquery”.

      • AIP_TRAINING_DATA_URI =”bigquery_destination.dataset_*.training”

      • AIP_VALIDATION_DATA_URI = “bigquery_destination.dataset_*.validation”

      • AIP_TEST_DATA_URI = “bigquery_destination.dataset_*.test”

    • args (List[Unions[str, *[int](https://python.readthedocs.io/en/latest/library/functions.html#int), [float](https://python.readthedocs.io/en/latest/library/functions.html#float)]*]) – Command line arguments to be passed to the Python script.

    • environment_variables (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Environment variables to be passed to the container. Should be a dictionary where keys are environment variable names and values are environment variable values for those names. At most 10 environment variables can be specified. The Name of the environment variable must be unique.

      environment_variables = {

      ‘MY_KEY’: ‘MY_VALUE’

      }

    • replica_count (int) – The number of worker replicas. If replica count = 1 then one chief replica will be provisioned. If replica_count > 1 the remainder will be provisioned as a worker replica pool.

    • machine_type (str) – The type of machine to use for training.

    • accelerator_type (str) – Hardware accelerator type. One of ACCELERATOR_TYPE_UNSPECIFIED, NVIDIA_TESLA_K80, NVIDIA_TESLA_P100, NVIDIA_TESLA_V100, NVIDIA_TESLA_P4, NVIDIA_TESLA_T4

    • accelerator_count (int) – The number of accelerators to attach to a worker replica.

    • boot_disk_type (str) – Type of the boot disk, default is pd-ssd. Valid values: pd-ssd (Persistent Disk Solid State Drive) or pd-standard (Persistent Disk Hard Disk Drive).

    • boot_disk_size_gb (int) – Size in GB of the boot disk, default is 100GB. boot disk size must be within the range of [100, 64000].

    • reduction_server_replica_count (int) – The number of reduction server replicas, default is 0.

    • reduction_server_machine_type (str) – Optional. The type of machine to use for reduction server.

    • reduction_server_container_uri (str) – Optional. The Uri of the reduction server container image. See details: https://cloud.google.com/vertex-ai/docs/training/distributed-training#reduce_training_time_with_reduction_server

    • training_fraction_split (float) – Optional. The fraction of the input data that is to be used to train the Model. This is ignored if Dataset is not provided.

    • validation_fraction_split (float) – Optional. The fraction of the input data that is to be used to validate the Model. This is ignored if Dataset is not provided.

    • test_fraction_split (float) – Optional. The fraction of the input data that is to be used to evaluate the Model. This is ignored if Dataset is not provided.

    • training_filter_split (str) – Optional. A filter on DataItems of the Dataset. DataItems that match this filter are used to train the Model. A filter with same syntax as the one used in DatasetService.ListDataItems may be used. If a single DataItem is matched by more than one of the FilterSplit filters, then it is assigned to the first set that applies to it in the training, validation, test order. This is ignored if Dataset is not provided.

    • validation_filter_split (str) – Optional. A filter on DataItems of the Dataset. DataItems that match this filter are used to validate the Model. A filter with same syntax as the one used in DatasetService.ListDataItems may be used. If a single DataItem is matched by more than one of the FilterSplit filters, then it is assigned to the first set that applies to it in the training, validation, test order. This is ignored if Dataset is not provided.

    • test_filter_split (str) – Optional. A filter on DataItems of the Dataset. DataItems that match this filter are used to test the Model. A filter with same syntax as the one used in DatasetService.ListDataItems may be used. If a single DataItem is matched by more than one of the FilterSplit filters, then it is assigned to the first set that applies to it in the training, validation, test order. This is ignored if Dataset is not provided.

    • predefined_split_column_name (str) – Optional. The key is a name of one of the Dataset’s data columns. The value of the key (either the label’s value or value in the column) must be one of {training, validation, test}, and it defines to which set the given piece of data is assigned. If for a piece of data the key is not present or has an invalid value, that piece is ignored by the pipeline.

      Supported only for tabular and time series Datasets.

    • timestamp_split_column_name (str) – Optional. The key is a name of one of the Dataset’s data columns. The value of the key values of the key (the values in the column) must be in RFC 3339 date-time format, where time-offset = “Z” (e.g. 1985-04-12T23:20:50.52Z). If for a piece of data the key is not present or has an invalid value, that piece is ignored by the pipeline.

      Supported only for tabular and time series Datasets.

    • timeout (int) – The maximum job running time in seconds. The default is 7 days.

    • restart_job_on_worker_restart (bool) – Restarts the entire CustomJob if a worker gets restarted. This feature can be used by distributed training jobs that are not resilient to workers leaving and joining a job.

    • enable_web_access (bool) – Whether you want Vertex AI to enable interactive shell access to training containers. https://cloud.google.com/vertex-ai/docs/training/monitor-debug-interactive-shell

    • enable_dashboard_access (bool) – Whether you want Vertex AI to enable access to the customized dashboard to training containers.

    • tensorboard (str) – Optional. The name of a Vertex AI [Tensorboard][google.cloud.aiplatform.v1beta1.Tensorboard] resource to which this CustomJob will upload Tensorboard logs. Format: projects/{project}/locations/{location}/tensorboards/{tensorboard}

      The training script should write Tensorboard to following Vertex AI environment variable:

      AIP_TENSORBOARD_LOG_DIR

      service_account is required with provided tensorboard. For more information on configuring your service account please visit: https://cloud.google.com/vertex-ai/docs/experiments/tensorboard-training

    • sync (bool) – Whether to execute this method synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

    • create_request_timeout (float) – Optional. The timeout for the create request in seconds.

    • disable_retries (bool) – Indicates if the job should retry for internal errors after the job starts running. If True, overrides restart_job_on_worker_restart to False.

    • persistent_resource_id (str) – Optional. The ID of the PersistentResource in the same Project and Location. If this is specified, the job will be run on existing machines held by the PersistentResource instead of on-demand short-live machines. The network, CMEK, and node pool configs on the job should be consistent with those on the PersistentResource, otherwise, the job will be rejected.

    • tpu_topology (str) – Optional. Specifies the tpu topology to be used for TPU training job. This field is required for TPU v5 versions. For details on the TPU topology, refer to https://cloud.google.com/tpu/docs/v5e#tpu-v5e-config. The topology must be a supported value for the TPU machine type.

  • Returns

    The trained Vertex AI Model resource or None if training did not

      produce a Vertex AI Model.

  • Return type

    model

property start_time(: Optional[datetime.datetime )

Time when the TrainingJob entered the PIPELINE_STATE_RUNNING for the first time.

property state(: Optional[[google.cloud.aiplatform_v1.types.pipeline_state.PipelineState](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.PipelineState)_ )

Current training state.

to_dict()

Returns the resource proto as a dictionary.

property update_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was last updated.

wait()

Helper method that blocks until all futures are complete.

wait_for_resource_creation()

Waits until resource has been created.

property web_access_uris(: Dict[str, str )

Get the web access uris of the backing custom job.

  • Returns

    Web access uris of the backing custom job.

  • Return type

    (Dict[str, str])

class google.cloud.aiplatform.CustomTrainingJob(display_name: str, script_path: str, container_uri: str, requirements: Optional[Sequence[str]] = None, model_serving_container_image_uri: Optional[str] = None, model_serving_container_predict_route: Optional[str] = None, model_serving_container_health_route: Optional[str] = None, model_serving_container_command: Optional[Sequence[str]] = None, model_serving_container_args: Optional[Sequence[str]] = None, model_serving_container_environment_variables: Optional[Dict[str, str]] = None, model_serving_container_ports: Optional[Sequence[int]] = None, model_description: Optional[str] = None, model_instance_schema_uri: Optional[str] = None, model_parameters_schema_uri: Optional[str] = None, model_prediction_schema_uri: Optional[str] = None, explanation_metadata: Optional[google.cloud.aiplatform_v1.types.explanation_metadata.ExplanationMetadata] = None, explanation_parameters: Optional[google.cloud.aiplatform_v1.types.explanation.ExplanationParameters] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None, labels: Optional[Dict[str, str]] = None, training_encryption_spec_key_name: Optional[str] = None, model_encryption_spec_key_name: Optional[str] = None, staging_bucket: Optional[str] = None)

Bases: google.cloud.aiplatform.training_jobs._CustomTrainingJob

Class to launch a Custom Training Job in Vertex AI using a script.

Takes a training implementation as a python script and executes that script in Cloud Vertex AI Training.

Constructs a Custom Training Job from a Python script.

job = aiplatform.CustomTrainingJob(

display_name=’test-train’,
script_path=’test_script.py’,
requirements=[‘pandas’, ‘numpy’],
container_uri=’gcr.io/cloud-aiplatform/training/tf-cpu.2-2:latest’,
model_serving_container_image_uri=’gcr.io/my-trainer/serving:1’,
model_serving_container_predict_route=’predict’,
model_serving_container_health_route=’metadata,
labels={‘key’: ‘value’},

)

Usage with Dataset:

ds = aiplatform.TabularDataset(

‘projects/my-project/locations/us-central1/datasets/12345’)

job.run(

ds,
replica_count=1,
model_display_name=’my-trained-model’,
model_labels={‘key’: ‘value’},

)

Usage without Dataset:

job.run(replica_count=1, model_display_name=’my-trained-model)

To ensure your model gets saved in Vertex AI, write your saved model to os.environ[“AIP_MODEL_DIR”] in your provided training script.

  • Parameters

    • display_name (str) – Required. The user-defined name of this TrainingPipeline.

    • script_path (str) – Required. Local path to training script.

    • container_uri (str) – Required: Uri of the training container image in the GCR.

    • requirements (Sequence[str]) – List of python packages dependencies of script.

    • model_serving_container_image_uri (str) – If the training produces a managed Vertex AI Model, the URI of the Model serving container suitable for serving the model produced by the training script.

    • model_serving_container_predict_route (str) – If the training produces a managed Vertex AI Model, An HTTP path to send prediction requests to the container, and which must be supported by it. If not specified a default HTTP path will be used by Vertex AI.

    • model_serving_container_health_route (str) – If the training produces a managed Vertex AI Model, an HTTP path to send health check requests to the container, and which must be supported by it. If not specified a standard HTTP path will be used by AI Platform.

    • model_serving_container_command (Sequence[str]) – The command with which the container is run. Not executed within a shell. The Docker image’s ENTRYPOINT is used if this is not provided. Variable references $(VAR_NAME) are expanded using the container’s environment. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not.

    • model_serving_container_args (Sequence[str]) – The arguments to the command. The Docker image’s CMD is used if this is not provided. Variable references $(VAR_NAME) are expanded using the container’s environment. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not.

    • model_serving_container_environment_variables (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – The environment variables that are to be present in the container. Should be a dictionary where keys are environment variable names and values are environment variable values for those names.

    • model_serving_container_ports (Sequence[int]) – Declaration of ports that are exposed by the container. This field is primarily informational, it gives Vertex AI information about the network connections the container uses. Listing or not a port here has no impact on whether the port is actually exposed, any port listening on the default “0.0.0.0” address inside a container will be accessible from the network.

    • model_description (str) – The description of the Model.

    • model_instance_schema_uri (str) – Optional. Points to a YAML file stored on Google Cloud Storage describing the format of a single instance, which are used in PredictRequest.instances, ExplainRequest.instances and BatchPredictionJob.input_config. The schema is defined as an OpenAPI 3.0.2 Schema Object. AutoML Models always have this field populated by AI Platform. Note: The URI given on output will be immutable and probably different, including the URI scheme, than the one given on input. The output URI will point to a location where the user only has a read access.

    • model_parameters_schema_uri (str) – Optional. Points to a YAML file stored on Google Cloud Storage describing the parameters of prediction and explanation via PredictRequest.parameters, ExplainRequest.parameters and BatchPredictionJob.model_parameters. The schema is defined as an OpenAPI 3.0.2 Schema Object. AutoML Models always have this field populated by AI Platform, if no parameters are supported it is set to an empty string. Note: The URI given on output will be immutable and probably different, including the URI scheme, than the one given on input. The output URI will point to a location where the user only has a read access.

    • model_prediction_schema_uri (str) – Optional. Points to a YAML file stored on Google Cloud Storage describing the format of a single prediction produced by this Model, which are returned via PredictResponse.predictions, ExplainResponse.explanations, and BatchPredictionJob.output_config. The schema is defined as an OpenAPI 3.0.2 Schema Object. AutoML Models always have this field populated by AI Platform. Note: The URI given on output will be immutable and probably different, including the URI scheme, than the one given on input. The output URI will point to a location where the user only has a read access.

    • explanation_metadata (explain.ExplanationMetadata) – Optional. Metadata describing the Model’s input and output for explanation. explanation_metadata is optional while explanation_parameters must be specified when used. For more details, see Ref docs http://tinyurl.com/1igh60kt

    • explanation_parameters (explain.ExplanationParameters) – Optional. Parameters to configure explaining for Model’s predictions. For more details, see Ref docs http://tinyurl.com/1an4zake

    • project (str) – Project to run training in. Overrides project set in aiplatform.init.

    • location (str) – Location to run training in. Overrides location set in aiplatform.init.

    • credentials (auth_credentials.Credentials) – Custom credentials to use to run call training service. Overrides credentials set in aiplatform.init.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The labels with user-defined metadata to organize TrainingPipelines. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information and examples of labels.

    • training_encryption_spec_key_name (Optional[str]) – Optional. The Cloud KMS resource identifier of the customer managed encryption key used to protect the training pipeline. Has the form: projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key. The key needs to be in the same region as where the compute resource is created.

      If set, this TrainingPipeline will be secured by this key.

      Note: Model trained by this TrainingPipeline is also secured by this key if model_to_upload is not set separately.

      Overrides encryption_spec_key_name set in aiplatform.init.

    • model_encryption_spec_key_name (Optional[str]) – Optional. The Cloud KMS resource identifier of the customer managed encryption key used to protect the model. Has the form: projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key. The key needs to be in the same region as where the compute resource is created.

      If set, the trained Model will be secured by this key.

      Overrides encryption_spec_key_name set in aiplatform.init.

    • staging_bucket (str) – Bucket used to stage source and training artifacts. Overrides staging_bucket set in aiplatform.init.

cancel()

Starts asynchronous cancellation on the TrainingJob. The server makes a best effort to cancel the job, but success is not guaranteed. On successful cancellation, the TrainingJob is not deleted; instead it becomes a job with state set to CANCELLED.

  • Raises

    RuntimeError – If this TrainingJob has not started running.

client_class()

alias of google.cloud.aiplatform.utils.PipelineClientWithOverride

property create_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was created.

delete(sync: bool = True)

Deletes this Vertex AI resource. WARNING: This deletion is permanent.

  • Parameters

    sync (bool) – Whether to execute this deletion synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

property display_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Display name of this resource.

done()

Method indicating whether a job has completed.

  • Returns

    True if the job has completed.

property encryption_spec(: Optional[[google.cloud.aiplatform_v1.types.encryption_spec.EncryptionSpec](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.EncryptionSpec)_ )

Customer-managed encryption key options for this Vertex AI resource.

If this is set, then all resources created by this Vertex AI resource will be encrypted with the provided encryption key.

property end_time(: Optional[datetime.datetime )

Time when the TrainingJob resource entered the PIPELINE_STATE_SUCCEEDED, PIPELINE_STATE_FAILED, PIPELINE_STATE_CANCELLED state.

property error(: Optional[google.rpc.status_pb2.Status )

Detailed error info for this TrainingJob resource. Only populated when the TrainingJob’s state is PIPELINE_STATE_FAILED or PIPELINE_STATE_CANCELLED.

property gca_resource(: [proto.message.Message](https://proto-plus-python.readthedocs.io/en/latest/reference/message.html#proto.message.Message )

The underlying resource proto representation.

classmethod get(resource_name: str, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Get Training Job for the given resource_name.

  • Parameters

    • resource_name (str) – Required. A fully-qualified resource name or ID.

    • project (str) – Optional project to retrieve training job from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional location to retrieve training job from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Custom credentials to use to upload this model. Overrides credentials set in aiplatform.init.

  • Raises

    ValueError – If the retrieved training job’s training task definition doesn’t match the custom training task definition.

  • Returns

    A Vertex AI Training Job

get_model(sync=True)

Vertex AI Model produced by this training, if one was produced.

  • Parameters

    sync (bool) – Whether to execute this method synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

  • Returns

    Vertex AI Model produced by this training

  • Return type

    model

  • Raises

    RuntimeError – If training failed or if a model was not produced by this training.

property has_failed(: [bool](https://python.readthedocs.io/en/latest/library/functions.html#bool )

Returns True if training has failed.

False otherwise.

property labels(: Dict[str, str )

User-defined labels containing metadata about this resource.

Read more about labels at https://goo.gl/xmQnxf

classmethod list(filter: Optional[str] = None, order_by: Optional[str] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

List all instances of this TrainingJob resource.

Example Usage:

aiplatform.CustomTrainingJob.list(

filter=’display_name=”experiment_a27”’,
order_by=’create_time desc’

)

  • Parameters

    • filter (str) – Optional. An expression for filtering the results of the request. For field names both snake_case and camelCase are supported.

    • order_by (str) – Optional. A comma-separated list of fields to order by, sorted in ascending order. Use “desc” after a field name for descending. Supported fields: display_name, create_time, update_time

    • project (str) – Optional. Project to retrieve list from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to retrieve list from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to retrieve list. Overrides credentials set in aiplatform.init.

  • Returns

    List[VertexAiResourceNoun] - A list of TrainingJob resource objects

property name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Name of this resource.

property network(: Optional[str )

The full name of the Google Compute Engine network to which this CustomTrainingJob should be peered.

Takes the format projects/{project}/global/networks/{network}. Where {project} is a project number, as in 12345, and {network} is a network name.

Private services access must already be configured for the network. If left unspecified, the CustomTrainingJob is not peered with any network.

property resource_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Full qualified resource name.

run(dataset: Optional[Union[google.cloud.aiplatform.datasets.image_dataset.ImageDataset, google.cloud.aiplatform.datasets.tabular_dataset.TabularDataset, google.cloud.aiplatform.datasets.text_dataset.TextDataset, google.cloud.aiplatform.datasets.video_dataset.VideoDataset]] = None, annotation_schema_uri: Optional[str] = None, model_display_name: Optional[str] = None, model_labels: Optional[Dict[str, str]] = None, model_id: Optional[str] = None, parent_model: Optional[str] = None, is_default_version: Optional[bool] = True, model_version_aliases: Optional[Sequence[str]] = None, model_version_description: Optional[str] = None, base_output_dir: Optional[str] = None, service_account: Optional[str] = None, network: Optional[str] = None, bigquery_destination: Optional[str] = None, args: Optional[List[Union[str, float, int]]] = None, environment_variables: Optional[Dict[str, str]] = None, replica_count: int = 1, machine_type: str = 'n1-standard-4', accelerator_type: str = 'ACCELERATOR_TYPE_UNSPECIFIED', accelerator_count: int = 0, boot_disk_type: str = 'pd-ssd', boot_disk_size_gb: int = 100, reduction_server_replica_count: int = 0, reduction_server_machine_type: Optional[str] = None, reduction_server_container_uri: Optional[str] = None, training_fraction_split: Optional[float] = None, validation_fraction_split: Optional[float] = None, test_fraction_split: Optional[float] = None, training_filter_split: Optional[str] = None, validation_filter_split: Optional[str] = None, test_filter_split: Optional[str] = None, predefined_split_column_name: Optional[str] = None, timestamp_split_column_name: Optional[str] = None, timeout: Optional[int] = None, restart_job_on_worker_restart: bool = False, enable_web_access: bool = False, enable_dashboard_access: bool = False, tensorboard: Optional[str] = None, sync=True, create_request_timeout: Optional[float] = None, disable_retries: bool = False, persistent_resource_id: Optional[str] = None, tpu_topology: Optional[str] = None)

Runs the custom training job.

Distributed Training Support: If replica count = 1 then one chief replica will be provisioned. If replica_count > 1 the remainder will be provisioned as a worker replica pool. ie: replica_count = 10 will result in 1 chief and 9 workers All replicas have same machine_type, accelerator_type, and accelerator_count

If training on a Vertex AI dataset, you can use one of the following split configurations:

Data fraction splits:
Any of `training_fraction_split`, `validation_fraction_split` and
`test_fraction_split` may optionally be provided, they must sum to up to 1. If
the provided ones sum to less than 1, the remainder is assigned to sets as
decided by Vertex AI. If none of the fractions are set, by default roughly 80%
of data will be used for training, 10% for validation, and 10% for test.

Data filter splits:
Assigns input data to training, validation, and test sets
based on the given filters, data pieces not matched by any
filter are ignored. Currently only supported for Datasets
containing DataItems.
If any of the filters in this message are to match nothing, then
they can be set as ‘-’ (the minus sign).
If using filter splits, all of `training_filter_split`, `validation_filter_split` and
`test_filter_split` must be provided.
Supported only for unstructured Datasets.

Predefined splits:
Assigns input data to training, validation, and test sets based on the value of a provided key.
If using predefined splits, `predefined_split_column_name` must be provided.
Supported only for tabular Datasets.

Timestamp splits:
Assigns input data to training, validation, and test sets
based on a provided timestamps. The youngest data pieces are
assigned to training set, next to validation set, and the oldest
to the test set.
Supported only for tabular Datasets.

  • Parameters

    • ( (dataset) – Union[

      datasets.ImageDataset, datasets.TabularDataset, datasets.TextDataset, datasets.VideoDataset,

      ]

    • ) – Vertex AI to fit this training against. Custom training script should retrieve datasets through passed in environment variables uris:

      os.environ[“AIP_TRAINING_DATA_URI”] os.environ[“AIP_VALIDATION_DATA_URI”] os.environ[“AIP_TEST_DATA_URI”]

      Additionally the dataset format is passed in as:

      os.environ[“AIP_DATA_FORMAT”]

    • annotation_schema_uri (str) – Google Cloud Storage URI points to a YAML file describing annotation schema. The schema is defined as an OpenAPI 3.0.2 Schema Object The schema files that can be used here are found in gs://google-cloud-aiplatform/schema/dataset/annotation/, note that the chosen schema must be consistent with metadata of the Dataset specified by dataset_id.

      Only Annotations that both match this schema and belong to DataItems not ignored by the split method are used in respectively training, validation or test role, depending on the role of the DataItem they are on.

      When used in conjunction with annotations_filter, the Annotations used for training are filtered by both annotations_filter and annotation_schema_uri.

    • model_display_name (str) – If the script produces a managed Vertex AI Model. The display name of the Model. The name can be up to 128 characters long and can be consist of any UTF-8 characters.

      If not provided upon creation, the job’s display_name is used.

    • model_labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The labels with user-defined metadata to organize your Models. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information and examples of labels.

    • model_id (str) – Optional. The ID to use for the Model produced by this job, which will become the final component of the model resource name. This value may be up to 63 characters, and valid characters are [a-z0-9_-]. The first character cannot be a number or hyphen.

    • parent_model (str) – Optional. The resource name or model ID of an existing model. The new model uploaded by this job will be a version of parent_model.

      Only set this field when training a new version of an existing model.

    • is_default_version (bool) – Optional. When set to True, the newly uploaded model version will automatically have alias “default” included. Subsequent uses of the model produced by this job without a version specified will use this “default” version.

      When set to False, the “default” alias will not be moved. Actions targeting the model version produced by this job will need to specifically reference this version by ID or alias.

      New model uploads, i.e. version 1, will always be “default” aliased.

    • model_version_aliases (Sequence[str]) – Optional. User provided version aliases so that the model version uploaded by this job can be referenced via alias instead of auto-generated version ID. A default version alias will be created for the first version of the model.

      The format is [a-z][a-zA-Z0-9-]{0,126}[a-z0-9]

    • model_version_description (str) – Optional. The description of the model version being uploaded by this job.

    • base_output_dir (str) – GCS output directory of job. If not provided a timestamped directory in the staging directory will be used.

      Vertex AI sets the following environment variables when it runs your training code:

      • AIP_MODEL_DIR: a Cloud Storage URI of a directory intended for saving model artifacts, i.e. <base_output_dir>/model/

      • AIP_CHECKPOINT_DIR: a Cloud Storage URI of a directory intended for saving checkpoints, i.e. <base_output_dir>/checkpoints/

      • AIP_TENSORBOARD_LOG_DIR: a Cloud Storage URI of a directory intended for saving TensorBoard logs, i.e. <base_output_dir>/logs/

    • service_account (str) – Specifies the service account for workload run-as account. Users submitting jobs must have act-as permission on this run-as account.

    • network (str) – The full name of the Compute Engine network to which the job should be peered. For example, projects/12345/global/networks/myVPC. Private services access must already be configured for the network. If left unspecified, the network set in aiplatform.init will be used. Otherwise, the job is not peered with any network.

    • bigquery_destination (str) – Provide this field if dataset is a BigQuery dataset. The BigQuery project location where the training data is to be written to. In the given project a new dataset is created with name dataset_<dataset-id>_<annotation-type>_<timestamp-of-training-call> where timestamp is in YYYY_MM_DDThh_mm_ss_sssZ format. All training input data will be written into that dataset. In the dataset three tables will be created, training, validation and test.

      • AIP_DATA_FORMAT = “bigquery”.

      • AIP_TRAINING_DATA_URI =”bigquery_destination.dataset_*.training”

      • AIP_VALIDATION_DATA_URI = “bigquery_destination.dataset_*.validation”

      • AIP_TEST_DATA_URI = “bigquery_destination.dataset_*.test”

    • args (List[Unions[str, *[int](https://python.readthedocs.io/en/latest/library/functions.html#int), [float](https://python.readthedocs.io/en/latest/library/functions.html#float)]*]) – Command line arguments to be passed to the Python script.

    • environment_variables (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Environment variables to be passed to the container. Should be a dictionary where keys are environment variable names and values are environment variable values for those names. At most 10 environment variables can be specified. The Name of the environment variable must be unique.

      environment_variables = {

      ‘MY_KEY’: ‘MY_VALUE’

      }

    • replica_count (int) – The number of worker replicas. If replica count = 1 then one chief replica will be provisioned. If replica_count > 1 the remainder will be provisioned as a worker replica pool.

    • machine_type (str) – The type of machine to use for training.

    • accelerator_type (str) – Hardware accelerator type. One of ACCELERATOR_TYPE_UNSPECIFIED, NVIDIA_TESLA_K80, NVIDIA_TESLA_P100, NVIDIA_TESLA_V100, NVIDIA_TESLA_P4, NVIDIA_TESLA_T4

    • accelerator_count (int) – The number of accelerators to attach to a worker replica.

    • boot_disk_type (str) – Type of the boot disk, default is pd-ssd. Valid values: pd-ssd (Persistent Disk Solid State Drive) or pd-standard (Persistent Disk Hard Disk Drive).

    • boot_disk_size_gb (int) – Size in GB of the boot disk, default is 100GB. boot disk size must be within the range of [100, 64000].

    • reduction_server_replica_count (int) – The number of reduction server replicas, default is 0.

    • reduction_server_machine_type (str) – Optional. The type of machine to use for reduction server.

    • reduction_server_container_uri (str) – Optional. The Uri of the reduction server container image. See details: https://cloud.google.com/vertex-ai/docs/training/distributed-training#reduce_training_time_with_reduction_server

    • training_fraction_split (float) – Optional. The fraction of the input data that is to be used to train the Model. This is ignored if Dataset is not provided.

    • validation_fraction_split (float) – Optional. The fraction of the input data that is to be used to validate the Model. This is ignored if Dataset is not provided.

    • test_fraction_split (float) – Optional. The fraction of the input data that is to be used to evaluate the Model. This is ignored if Dataset is not provided.

    • training_filter_split (str) – Optional. A filter on DataItems of the Dataset. DataItems that match this filter are used to train the Model. A filter with same syntax as the one used in DatasetService.ListDataItems may be used. If a single DataItem is matched by more than one of the FilterSplit filters, then it is assigned to the first set that applies to it in the training, validation, test order. This is ignored if Dataset is not provided.

    • validation_filter_split (str) – Optional. A filter on DataItems of the Dataset. DataItems that match this filter are used to validate the Model. A filter with same syntax as the one used in DatasetService.ListDataItems may be used. If a single DataItem is matched by more than one of the FilterSplit filters, then it is assigned to the first set that applies to it in the training, validation, test order. This is ignored if Dataset is not provided.

    • test_filter_split (str) – Optional. A filter on DataItems of the Dataset. DataItems that match this filter are used to test the Model. A filter with same syntax as the one used in DatasetService.ListDataItems may be used. If a single DataItem is matched by more than one of the FilterSplit filters, then it is assigned to the first set that applies to it in the training, validation, test order. This is ignored if Dataset is not provided.

    • predefined_split_column_name (str) – Optional. The key is a name of one of the Dataset’s data columns. The value of the key (either the label’s value or value in the column) must be one of {training, validation, test}, and it defines to which set the given piece of data is assigned. If for a piece of data the key is not present or has an invalid value, that piece is ignored by the pipeline.

      Supported only for tabular and time series Datasets.

    • timestamp_split_column_name (str) – Optional. The key is a name of one of the Dataset’s data columns. The value of the key values of the key (the values in the column) must be in RFC 3339 date-time format, where time-offset = “Z” (e.g. 1985-04-12T23:20:50.52Z). If for a piece of data the key is not present or has an invalid value, that piece is ignored by the pipeline.

      Supported only for tabular and time series Datasets.

    • timeout (int) – The maximum job running time in seconds. The default is 7 days.

    • restart_job_on_worker_restart (bool) – Restarts the entire CustomJob if a worker gets restarted. This feature can be used by distributed training jobs that are not resilient to workers leaving and joining a job.

    • enable_web_access (bool) – Whether you want Vertex AI to enable interactive shell access to training containers. https://cloud.google.com/vertex-ai/docs/training/monitor-debug-interactive-shell

    • enable_dashboard_access (bool) – Whether you want Vertex AI to enable access to the customized dashboard to training containers.

    • tensorboard (str) – Optional. The name of a Vertex AI [Tensorboard][google.cloud.aiplatform.v1beta1.Tensorboard] resource to which this CustomJob will upload Tensorboard logs. Format: projects/{project}/locations/{location}/tensorboards/{tensorboard}

      The training script should write Tensorboard to following Vertex AI environment variable:

      AIP_TENSORBOARD_LOG_DIR

      service_account is required with provided tensorboard. For more information on configuring your service account please visit: https://cloud.google.com/vertex-ai/docs/experiments/tensorboard-training

    • create_request_timeout (float) – Optional. The timeout for the create request in seconds.

    • sync (bool) – Whether to execute this method synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

    • disable_retries (bool) – Indicates if the job should retry for internal errors after the job starts running. If True, overrides restart_job_on_worker_restart to False.

    • persistent_resource_id (str) – Optional. The ID of the PersistentResource in the same Project and Location. If this is specified, the job will be run on existing machines held by the PersistentResource instead of on-demand short-live machines. The network, CMEK, and node pool configs on the job should be consistent with those on the PersistentResource, otherwise, the job will be rejected.

    • tpu_topology (str) – Optional. Specifies the tpu topology to be used for TPU training job. This field is required for TPU v5 versions. For details on the TPU topology, refer to https://cloud.google.com/tpu/docs/v5e#tpu-v5e-config. The topology must be a supported value for the TPU machine type.

  • Returns

    The trained Vertex AI Model resource or None if training did not

      produce a Vertex AI Model.

  • Return type

    model

property start_time(: Optional[datetime.datetime )

Time when the TrainingJob entered the PIPELINE_STATE_RUNNING for the first time.

property state(: Optional[[google.cloud.aiplatform_v1.types.pipeline_state.PipelineState](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.PipelineState)_ )

Current training state.

submit(dataset: Optional[Union[google.cloud.aiplatform.datasets.image_dataset.ImageDataset, google.cloud.aiplatform.datasets.tabular_dataset.TabularDataset, google.cloud.aiplatform.datasets.text_dataset.TextDataset, google.cloud.aiplatform.datasets.video_dataset.VideoDataset]] = None, annotation_schema_uri: Optional[str] = None, model_display_name: Optional[str] = None, model_labels: Optional[Dict[str, str]] = None, model_id: Optional[str] = None, parent_model: Optional[str] = None, is_default_version: Optional[bool] = True, model_version_aliases: Optional[Sequence[str]] = None, model_version_description: Optional[str] = None, base_output_dir: Optional[str] = None, service_account: Optional[str] = None, network: Optional[str] = None, bigquery_destination: Optional[str] = None, args: Optional[List[Union[str, float, int]]] = None, environment_variables: Optional[Dict[str, str]] = None, replica_count: int = 1, machine_type: str = 'n1-standard-4', accelerator_type: str = 'ACCELERATOR_TYPE_UNSPECIFIED', accelerator_count: int = 0, boot_disk_type: str = 'pd-ssd', boot_disk_size_gb: int = 100, reduction_server_replica_count: int = 0, reduction_server_machine_type: Optional[str] = None, reduction_server_container_uri: Optional[str] = None, training_fraction_split: Optional[float] = None, validation_fraction_split: Optional[float] = None, test_fraction_split: Optional[float] = None, training_filter_split: Optional[str] = None, validation_filter_split: Optional[str] = None, test_filter_split: Optional[str] = None, predefined_split_column_name: Optional[str] = None, timestamp_split_column_name: Optional[str] = None, timeout: Optional[int] = None, restart_job_on_worker_restart: bool = False, enable_web_access: bool = False, enable_dashboard_access: bool = False, tensorboard: Optional[str] = None, sync=True, create_request_timeout: Optional[float] = None, disable_retries: bool = False, persistent_resource_id: Optional[str] = None, tpu_topology: Optional[str] = None)

Submits the custom training job without blocking until completion.

Distributed Training Support: If replica count = 1 then one chief replica will be provisioned. If replica_count > 1 the remainder will be provisioned as a worker replica pool. ie: replica_count = 10 will result in 1 chief and 9 workers All replicas have same machine_type, accelerator_type, and accelerator_count

If training on a Vertex AI dataset, you can use one of the following split configurations:

Data fraction splits:
Any of `training_fraction_split`, `validation_fraction_split` and
`test_fraction_split` may optionally be provided, they must sum to up to 1. If
the provided ones sum to less than 1, the remainder is assigned to sets as
decided by Vertex AI. If none of the fractions are set, by default roughly 80%
of data will be used for training, 10% for validation, and 10% for test.

Data filter splits:
Assigns input data to training, validation, and test sets
based on the given filters, data pieces not matched by any
filter are ignored. Currently only supported for Datasets
containing DataItems.
If any of the filters in this message are to match nothing, then
they can be set as ‘-’ (the minus sign).
If using filter splits, all of `training_filter_split`, `validation_filter_split` and
`test_filter_split` must be provided.
Supported only for unstructured Datasets.

Predefined splits:
Assigns input data to training, validation, and test sets based on the value of a provided key.
If using predefined splits, `predefined_split_column_name` must be provided.
Supported only for tabular Datasets.

Timestamp splits:
Assigns input data to training, validation, and test sets
based on a provided timestamps. The youngest data pieces are
assigned to training set, next to validation set, and the oldest
to the test set.
Supported only for tabular Datasets.

  • Parameters

    • ( (dataset) – Union[

      datasets.ImageDataset, datasets.TabularDataset, datasets.TextDataset, datasets.VideoDataset,

      ]

    • ) – Vertex AI to fit this training against. Custom training script should retrieve datasets through passed in environment variables uris:

      os.environ[“AIP_TRAINING_DATA_URI”] os.environ[“AIP_VALIDATION_DATA_URI”] os.environ[“AIP_TEST_DATA_URI”]

      Additionally the dataset format is passed in as:

      os.environ[“AIP_DATA_FORMAT”]

    • annotation_schema_uri (str) – Google Cloud Storage URI points to a YAML file describing annotation schema. The schema is defined as an OpenAPI 3.0.2 Schema Object The schema files that can be used here are found in gs://google-cloud-aiplatform/schema/dataset/annotation/, note that the chosen schema must be consistent with metadata of the Dataset specified by dataset_id.

      Only Annotations that both match this schema and belong to DataItems not ignored by the split method are used in respectively training, validation or test role, depending on the role of the DataItem they are on.

      When used in conjunction with annotations_filter, the Annotations used for training are filtered by both annotations_filter and annotation_schema_uri.

    • model_display_name (str) – If the script produces a managed Vertex AI Model. The display name of the Model. The name can be up to 128 characters long and can be consist of any UTF-8 characters.

      If not provided upon creation, the job’s display_name is used.

    • model_labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The labels with user-defined metadata to organize your Models. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information and examples of labels.

    • model_id (str) – Optional. The ID to use for the Model produced by this job, which will become the final component of the model resource name. This value may be up to 63 characters, and valid characters are [a-z0-9_-]. The first character cannot be a number or hyphen.

    • parent_model (str) – Optional. The resource name or model ID of an existing model. The new model uploaded by this job will be a version of parent_model.

      Only set this field when training a new version of an existing model.

    • is_default_version (bool) – Optional. When set to True, the newly uploaded model version will automatically have alias “default” included. Subsequent uses of the model produced by this job without a version specified will use this “default” version.

      When set to False, the “default” alias will not be moved. Actions targeting the model version produced by this job will need to specifically reference this version by ID or alias.

      New model uploads, i.e. version 1, will always be “default” aliased.

    • model_version_aliases (Sequence[str]) – Optional. User provided version aliases so that the model version uploaded by this job can be referenced via alias instead of auto-generated version ID. A default version alias will be created for the first version of the model.

      The format is [a-z][a-zA-Z0-9-]{0,126}[a-z0-9]

    • model_version_description (str) – Optional. The description of the model version being uploaded by this job.

    • base_output_dir (str) – GCS output directory of job. If not provided a timestamped directory in the staging directory will be used.

      Vertex AI sets the following environment variables when it runs your training code:

      • AIP_MODEL_DIR: a Cloud Storage URI of a directory intended for saving model artifacts, i.e. <base_output_dir>/model/

      • AIP_CHECKPOINT_DIR: a Cloud Storage URI of a directory intended for saving checkpoints, i.e. <base_output_dir>/checkpoints/

      • AIP_TENSORBOARD_LOG_DIR: a Cloud Storage URI of a directory intended for saving TensorBoard logs, i.e. <base_output_dir>/logs/

    • service_account (str) – Specifies the service account for workload run-as account. Users submitting jobs must have act-as permission on this run-as account.

    • network (str) – The full name of the Compute Engine network to which the job should be peered. For example, projects/12345/global/networks/myVPC. Private services access must already be configured for the network. If left unspecified, the network set in aiplatform.init will be used. Otherwise, the job is not peered with any network.

    • bigquery_destination (str) – Provide this field if dataset is a BigQuery dataset. The BigQuery project location where the training data is to be written to. In the given project a new dataset is created with name dataset_<dataset-id>_<annotation-type>_<timestamp-of-training-call> where timestamp is in YYYY_MM_DDThh_mm_ss_sssZ format. All training input data will be written into that dataset. In the dataset three tables will be created, training, validation and test.

      • AIP_DATA_FORMAT = “bigquery”.

      • AIP_TRAINING_DATA_URI =”bigquery_destination.dataset_*.training”

      • AIP_VALIDATION_DATA_URI = “bigquery_destination.dataset_*.validation”

      • AIP_TEST_DATA_URI = “bigquery_destination.dataset_*.test”

    • args (List[Unions[str, *[int](https://python.readthedocs.io/en/latest/library/functions.html#int), [float](https://python.readthedocs.io/en/latest/library/functions.html#float)]*]) – Command line arguments to be passed to the Python script.

    • environment_variables (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Environment variables to be passed to the container. Should be a dictionary where keys are environment variable names and values are environment variable values for those names. At most 10 environment variables can be specified. The Name of the environment variable must be unique.

      environment_variables = {

      ‘MY_KEY’: ‘MY_VALUE’

      }

    • replica_count (int) – The number of worker replicas. If replica count = 1 then one chief replica will be provisioned. If replica_count > 1 the remainder will be provisioned as a worker replica pool.

    • machine_type (str) – The type of machine to use for training.

    • accelerator_type (str) – Hardware accelerator type. One of ACCELERATOR_TYPE_UNSPECIFIED, NVIDIA_TESLA_K80, NVIDIA_TESLA_P100, NVIDIA_TESLA_V100, NVIDIA_TESLA_P4, NVIDIA_TESLA_T4

    • accelerator_count (int) – The number of accelerators to attach to a worker replica.

    • boot_disk_type (str) – Type of the boot disk, default is pd-ssd. Valid values: pd-ssd (Persistent Disk Solid State Drive) or pd-standard (Persistent Disk Hard Disk Drive).

    • boot_disk_size_gb (int) – Size in GB of the boot disk, default is 100GB. boot disk size must be within the range of [100, 64000].

    • reduction_server_replica_count (int) – The number of reduction server replicas, default is 0.

    • reduction_server_machine_type (str) – Optional. The type of machine to use for reduction server.

    • reduction_server_container_uri (str) – Optional. The Uri of the reduction server container image. See details: https://cloud.google.com/vertex-ai/docs/training/distributed-training#reduce_training_time_with_reduction_server

    • training_fraction_split (float) – Optional. The fraction of the input data that is to be used to train the Model. This is ignored if Dataset is not provided.

    • validation_fraction_split (float) – Optional. The fraction of the input data that is to be used to validate the Model. This is ignored if Dataset is not provided.

    • test_fraction_split (float) – Optional. The fraction of the input data that is to be used to evaluate the Model. This is ignored if Dataset is not provided.

    • training_filter_split (str) – Optional. A filter on DataItems of the Dataset. DataItems that match this filter are used to train the Model. A filter with same syntax as the one used in DatasetService.ListDataItems may be used. If a single DataItem is matched by more than one of the FilterSplit filters, then it is assigned to the first set that applies to it in the training, validation, test order. This is ignored if Dataset is not provided.

    • validation_filter_split (str) – Optional. A filter on DataItems of the Dataset. DataItems that match this filter are used to validate the Model. A filter with same syntax as the one used in DatasetService.ListDataItems may be used. If a single DataItem is matched by more than one of the FilterSplit filters, then it is assigned to the first set that applies to it in the training, validation, test order. This is ignored if Dataset is not provided.

    • test_filter_split (str) – Optional. A filter on DataItems of the Dataset. DataItems that match this filter are used to test the Model. A filter with same syntax as the one used in DatasetService.ListDataItems may be used. If a single DataItem is matched by more than one of the FilterSplit filters, then it is assigned to the first set that applies to it in the training, validation, test order. This is ignored if Dataset is not provided.

    • predefined_split_column_name (str) – Optional. The key is a name of one of the Dataset’s data columns. The value of the key (either the label’s value or value in the column) must be one of {training, validation, test}, and it defines to which set the given piece of data is assigned. If for a piece of data the key is not present or has an invalid value, that piece is ignored by the pipeline.

      Supported only for tabular and time series Datasets.

    • timestamp_split_column_name (str) – Optional. The key is a name of one of the Dataset’s data columns. The value of the key values of the key (the values in the column) must be in RFC 3339 date-time format, where time-offset = “Z” (e.g. 1985-04-12T23:20:50.52Z). If for a piece of data the key is not present or has an invalid value, that piece is ignored by the pipeline.

      Supported only for tabular and time series Datasets.

    • timeout (int) – The maximum job running time in seconds. The default is 7 days.

    • restart_job_on_worker_restart (bool) – Restarts the entire CustomJob if a worker gets restarted. This feature can be used by distributed training jobs that are not resilient to workers leaving and joining a job.

    • enable_web_access (bool) – Whether you want Vertex AI to enable interactive shell access to training containers. https://cloud.google.com/vertex-ai/docs/training/monitor-debug-interactive-shell

    • enable_dashboard_access (bool) – Whether you want Vertex AI to enable access to the customized dashboard to training containers.

    • tensorboard (str) – Optional. The name of a Vertex AI [Tensorboard][google.cloud.aiplatform.v1beta1.Tensorboard] resource to which this CustomJob will upload Tensorboard logs. Format: projects/{project}/locations/{location}/tensorboards/{tensorboard}

      The training script should write Tensorboard to following Vertex AI environment variable:

      AIP_TENSORBOARD_LOG_DIR

      service_account is required with provided tensorboard. For more information on configuring your service account please visit: https://cloud.google.com/vertex-ai/docs/experiments/tensorboard-training

    • create_request_timeout (float) – Optional. The timeout for the create request in seconds.

    • sync (bool) – Whether to execute this method synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

    • disable_retries (bool) – Indicates if the job should retry for internal errors after the job starts running. If True, overrides restart_job_on_worker_restart to False.

    • persistent_resource_id (str) – Optional. The ID of the PersistentResource in the same Project and Location. If this is specified, the job will be run on existing machines held by the PersistentResource instead of on-demand short-live machines. The network, CMEK, and node pool configs on the job should be consistent with those on the PersistentResource, otherwise, the job will be rejected.

    • tpu_topology (str) – Optional. Specifies the tpu topology to be used for TPU training job. This field is required for TPU v5 versions. For details on the TPU topology, refer to https://cloud.google.com/tpu/docs/v5e#tpu-v5e-config. The topology must be a supported value for the TPU machine type.

  • Returns

    The trained Vertex AI Model resource or None if training did not

      produce a Vertex AI Model.

  • Return type

    model

to_dict()

Returns the resource proto as a dictionary.

property update_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was last updated.

wait()

Helper method that blocks until all futures are complete.

wait_for_resource_creation()

Waits until resource has been created.

property web_access_uris(: Dict[str, str )

Get the web access uris of the backing custom job.

  • Returns

    Web access uris of the backing custom job.

  • Return type

    (Dict[str, str])

class google.cloud.aiplatform.Endpoint(endpoint_name: str, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Bases: google.cloud.aiplatform.base.VertexAiResourceNounWithFutureManager, google.cloud.aiplatform.base.PreviewMixin

Retrieves an endpoint resource.

  • Parameters

    • endpoint_name (str) – Required. A fully-qualified endpoint resource name or endpoint ID. Example: “projects/123/locations/us-central1/endpoints/456” or “456” when project and location are initialized or passed.

    • project (str) – Optional. Project to retrieve endpoint from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to retrieve endpoint from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to upload this model. Overrides credentials set in aiplatform.init.

client_class()

alias of google.cloud.aiplatform.utils.EndpointClientWithOverride

classmethod create(display_name: Optional[str] = None, description: Optional[str] = None, labels: Optional[Dict[str, str]] = None, metadata: Optional[Sequence[Tuple[str, str]]] = (), project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None, encryption_spec_key_name: Optional[str] = None, sync=True, create_request_timeout: Optional[float] = None, endpoint_id: Optional[str] = None, enable_request_response_logging=False, request_response_logging_sampling_rate: Optional[float] = None, request_response_logging_bq_destination_table: Optional[str] = None)

Creates a new endpoint.

  • Parameters

    • display_name (str) – Optional. The user-defined name of the Endpoint. The name can be up to 128 characters long and can be consist of any UTF-8 characters.

    • description (str) – Optional. The description of the Endpoint.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The labels with user-defined metadata to organize your Endpoints. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information and examples of labels.

    • metadata (Sequence[Tuple[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*]) – Optional. Strings which should be sent along with the request as metadata.

    • project (str) – Optional. Project to retrieve endpoint from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to retrieve endpoint from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to upload this model. Overrides credentials set in aiplatform.init.

    • encryption_spec_key_name (str) – Optional. The Cloud KMS resource identifier of the customer managed encryption key used to protect the model. Has the form: projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key. The key needs to be in the same region as where the compute resource is created.

      If set, this Endpoint and all sub-resources of this Endpoint will be secured by this key.

      Overrides encryption_spec_key_name set in aiplatform.init.

    • sync (bool) – Whether to execute this method synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

    • create_request_timeout (float) – Optional. The timeout for the create request in seconds.

    • endpoint_id (str) – Optional. The ID to use for endpoint, which will become the final component of the endpoint resource name. If not provided, Vertex AI will generate a value for this ID.

      This value should be 1-10 characters, and valid characters are /[0-9]/. When using HTTP/JSON, this field is populated based on a query string argument, such as ?endpoint_id=12345. This is the fallback for fields that are not included in either the URI or the body.

    • enable_request_response_logging (bool) – Optional. Whether to enable request & response logging for this endpoint.

    • request_response_logging_sampling_rate (float) – Optional. The request response logging sampling rate. If not set, default is 0.0.

    • request_response_logging_bq_destination_table (str) – Optional. The request response logging bigquery destination. If not set, will create a table with name: bq://{project_id}.logging_{endpoint_display_name}_{endpoint_id}.request_response_logging.

  • Returns

    Created endpoint.

  • Return type

    endpoint (aiplatform.Endpoint)

property create_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was created.

delete(force: bool = False, sync: bool = True)

Deletes this Vertex AI Endpoint resource. If force is set to True, all models on this Endpoint will be undeployed prior to deletion.

  • Parameters

    • force (bool) – Required. If force is set to True, all deployed models on this Endpoint will be undeployed first. Default is False.

    • sync (bool) – Whether to execute this method synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

  • Raises

    FailedPrecondition – If models are deployed on this Endpoint and force = False.

deploy(model: google.cloud.aiplatform.models.Model, deployed_model_display_name: Optional[str] = None, traffic_percentage: int = 0, traffic_split: Optional[Dict[str, int]] = None, machine_type: Optional[str] = None, min_replica_count: int = 1, max_replica_count: int = 1, accelerator_type: Optional[str] = None, accelerator_count: Optional[int] = None, tpu_topology: Optional[str] = None, service_account: Optional[str] = None, explanation_metadata: Optional[google.cloud.aiplatform_v1.types.explanation_metadata.ExplanationMetadata] = None, explanation_parameters: Optional[google.cloud.aiplatform_v1.types.explanation.ExplanationParameters] = None, metadata: Optional[Sequence[Tuple[str, str]]] = (), sync=True, deploy_request_timeout: Optional[float] = None, autoscaling_target_cpu_utilization: Optional[int] = None, autoscaling_target_accelerator_duty_cycle: Optional[int] = None, enable_access_logging=False, disable_container_logging: bool = False)

Deploys a Model to the Endpoint.

  • Parameters

    • model (aiplatform.Model) – Required. Model to be deployed.

    • deployed_model_display_name (str) – Optional. The display name of the DeployedModel. If not provided upon creation, the Model’s display_name is used.

    • traffic_percentage (int) – Optional. Desired traffic to newly deployed model. Defaults to 0 if there are pre-existing deployed models. Defaults to 100 if there are no pre-existing deployed models. Negative values should not be provided. Traffic of previously deployed models at the endpoint will be scaled down to accommodate new deployed model’s traffic. Should not be provided if traffic_split is provided.

    • traffic_split (Dict[str, *[int](https://python.readthedocs.io/en/latest/library/functions.html#int)]*) – Optional. A map from a DeployedModel’s ID to the percentage of this Endpoint’s traffic that should be forwarded to that DeployedModel. If a DeployedModel’s ID is not listed in this map, then it receives no traffic. The traffic percentage values must add up to 100, or map must be empty if the Endpoint is to not accept any traffic at the moment. Key for model being deployed is “0”. Should not be provided if traffic_percentage is provided.

    • machine_type (str) – Optional. The type of machine. Not specifying machine type will result in model to be deployed with automatic resources.

    • min_replica_count (int) – Optional. The minimum number of machine replicas this deployed model will be always deployed on. If traffic against it increases, it may dynamically be deployed onto more replicas, and as traffic decreases, some of these extra replicas may be freed.

    • max_replica_count (int) – Optional. The maximum number of replicas this deployed model may be deployed on when the traffic against it increases. If requested value is too large, the deployment will error, but if deployment succeeds then the ability to scale the model to that many replicas is guaranteed (barring service outages). If traffic against the deployed model increases beyond what its replicas at maximum may handle, a portion of the traffic will be dropped. If this value is not provided, the larger value of min_replica_count or 1 will be used. If value provided is smaller than min_replica_count, it will automatically be increased to be min_replica_count.

    • accelerator_type (str) – Optional. Hardware accelerator type. Must also set accelerator_count if used. One of ACCELERATOR_TYPE_UNSPECIFIED, NVIDIA_TESLA_K80, NVIDIA_TESLA_P100, NVIDIA_TESLA_V100, NVIDIA_TESLA_P4, NVIDIA_TESLA_T4

    • accelerator_count (int) – Optional. The number of accelerators to attach to a worker replica.

    • tpu_topology (str) – Optional. The TPU topology to use for the DeployedModel. Required for CloudTPU multihost deployments.

    • service_account (str) – The service account that the DeployedModel’s container runs as. Specify the email address of the service account. If this service account is not specified, the container runs as a service account that doesn’t have access to the resource project. Users deploying the Model must have the iam.serviceAccounts.actAs permission on this service account.

    • explanation_metadata (aiplatform.explain.ExplanationMetadata) – Optional. Metadata describing the Model’s input and output for explanation. explanation_metadata is optional while explanation_parameters must be specified when used. For more details, see Ref docs http://tinyurl.com/1igh60kt

    • explanation_parameters (aiplatform.explain.ExplanationParameters) – Optional. Parameters to configure explaining for Model’s predictions. For more details, see Ref docs http://tinyurl.com/1an4zake

    • metadata (Sequence[Tuple[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*]) – Optional. Strings which should be sent along with the request as metadata.

    • sync (bool) – Whether to execute this method synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

    • deploy_request_timeout (float) – Optional. The timeout for the deploy request in seconds.

    • autoscaling_target_cpu_utilization (int) – Target CPU Utilization to use for Autoscaling Replicas. A default value of 60 will be used if not specified.

    • autoscaling_target_accelerator_duty_cycle (int) – Target Accelerator Duty Cycle. Must also set accelerator_type and accelerator_count if specified. A default value of 60 will be used if not specified.

    • enable_access_logging (bool) – Whether to enable endpoint access logging. Defaults to False.

    • disable_container_logging (bool) – If True, container logs from the deployed model will not be written to Cloud Logging. Defaults to False.

property display_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Display name of this resource.

property encryption_spec(: Optional[[google.cloud.aiplatform_v1.types.encryption_spec.EncryptionSpec](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.EncryptionSpec)_ )

Customer-managed encryption key options for this Vertex AI resource.

If this is set, then all resources created by this Vertex AI resource will be encrypted with the provided encryption key.

explain(instances: List[Dict], parameters: Optional[Dict] = None, deployed_model_id: Optional[str] = None, timeout: Optional[float] = None)

Make a prediction with explanations against this Endpoint.

Example usage:

response = my_endpoint.explain(instances=[…])
my_explanations = response.explanations

  • Parameters

    • instances (List) – Required. The instances that are the input to the prediction call. A DeployedModel may have an upper limit on the number of instances it supports per request, and when it is exceeded the prediction call errors in case of AutoML Models, or, in case of customer created Models, the behaviour is as documented by that Model. The schema of any single instance may be specified via Endpoint’s DeployedModels’ [Model’s][google.cloud.aiplatform.v1beta1.DeployedModel.model] [PredictSchemata’s][google.cloud.aiplatform.v1beta1.Model.predict_schemata] instance_schema_uri.

    • parameters (Dict) – The parameters that govern the prediction. The schema of the parameters may be specified via Endpoint’s DeployedModels’ [Model’s ][google.cloud.aiplatform.v1beta1.DeployedModel.model] [PredictSchemata’s][google.cloud.aiplatform.v1beta1.Model.predict_schemata] parameters_schema_uri.

    • deployed_model_id (str) – Optional. If specified, this ExplainRequest will be served by the chosen DeployedModel, overriding this Endpoint’s traffic split.

    • timeout (float) – Optional. The timeout for this request in seconds.

  • Returns

    Prediction with returned predictions, explanations, and Model ID.

  • Return type

    prediction (aiplatform.Prediction)

async explain_async(instances: List[Dict], *, parameters: Optional[Dict] = None, deployed_model_id: Optional[str] = None, timeout: Optional[float] = None)

Make a prediction with explanations against this Endpoint.

Example usage:

`\`
response = await my_endpoint.explain_async(instances=[...])
my_explanations = response.explanations
\``

  • Parameters

    • instances (List) – Required. The instances that are the input to the prediction call. A DeployedModel may have an upper limit on the number of instances it supports per request, and when it is exceeded the prediction call errors in case of AutoML Models, or, in case of customer created Models, the behaviour is as documented by that Model. The schema of any single instance may be specified via Endpoint’s DeployedModels’ [Model’s][google.cloud.aiplatform.v1beta1.DeployedModel.model] [PredictSchemata’s][google.cloud.aiplatform.v1beta1.Model.predict_schemata] instance_schema_uri.

    • parameters (Dict) – The parameters that govern the prediction. The schema of the parameters may be specified via Endpoint’s DeployedModels’ [Model’s ][google.cloud.aiplatform.v1beta1.DeployedModel.model] [PredictSchemata’s][google.cloud.aiplatform.v1beta1.Model.predict_schemata] parameters_schema_uri.

    • deployed_model_id (str) – Optional. If specified, this ExplainRequest will be served by the chosen DeployedModel, overriding this Endpoint’s traffic split.

    • timeout (float) – Optional. The timeout for this request in seconds.

  • Returns

    Prediction with returned predictions, explanations, and Model ID.

  • Return type

    prediction (aiplatform.Prediction)

property gca_resource(: [proto.message.Message](https://proto-plus-python.readthedocs.io/en/latest/reference/message.html#proto.message.Message )

The underlying resource proto representation.

property labels(: Dict[str, str )

User-defined labels containing metadata about this resource.

Read more about labels at https://goo.gl/xmQnxf

classmethod list(filter: Optional[str] = None, order_by: Optional[str] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

List all Endpoint resource instances.

Example Usage:

aiplatform.Endpoint.list(

    filter=’labels.my_label=”my_label_value” OR display_name=!”old_endpoint”’,

)

  • Parameters

    • filter (str) – Optional. An expression for filtering the results of the request. For field names both snake_case and camelCase are supported.

    • order_by (str) – Optional. A comma-separated list of fields to order by, sorted in ascending order. Use “desc” after a field name for descending. Supported fields: display_name, create_time, update_time

    • project (str) – Optional. Project to retrieve list from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to retrieve list from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to retrieve list. Overrides credentials set in aiplatform.init.

  • Returns

    A list of Endpoint resource objects

  • Return type

    List[models.Endpoint]

list_models()

Returns a list of the models deployed to this Endpoint.

  • Returns

    A list of the models deployed in this Endpoint.

  • Return type

    deployed_models (List[aiplatform.gapic.DeployedModel])

property name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Name of this resource.

property network(: Optional[str )

The full name of the Google Compute Engine network to which this Endpoint should be peered.

Takes the format projects/{project}/global/networks/{network}. Where {project} is a project number, as in 12345, and {network} is a network name.

Private services access must already be configured for the network. If left unspecified, the Endpoint is not peered with any network.

predict(instances: List, parameters: Optional[Dict] = None, timeout: Optional[float] = None, use_raw_predict: Optional[bool] = False)

Make a prediction against this Endpoint.

  • Parameters

    • instances (List) – Required. The instances that are the input to the prediction call. A DeployedModel may have an upper limit on the number of instances it supports per request, and when it is exceeded the prediction call errors in case of AutoML Models, or, in case of customer created Models, the behaviour is as documented by that Model. The schema of any single instance may be specified via Endpoint’s DeployedModels’ [Model’s][google.cloud.aiplatform.v1beta1.DeployedModel.model] [PredictSchemata’s][google.cloud.aiplatform.v1beta1.Model.predict_schemata] instance_schema_uri.

    • parameters (Dict) – The parameters that govern the prediction. The schema of the parameters may be specified via Endpoint’s DeployedModels’ [Model’s ][google.cloud.aiplatform.v1beta1.DeployedModel.model] [PredictSchemata’s][google.cloud.aiplatform.v1beta1.Model.predict_schemata] parameters_schema_uri.

    • timeout (float) – Optional. The timeout for this request in seconds.

    • use_raw_predict (bool) – Optional. Default value is False. If set to True, the underlying prediction call will be made against Endpoint.raw_predict().

  • Returns

    Prediction with returned predictions and Model ID.

  • Return type

    prediction (aiplatform.Prediction)

async predict_async(instances: List, *, parameters: Optional[Dict] = None, timeout: Optional[float] = None)

Make an asynchronous prediction against this Endpoint. Example usage:

\ response = await my_endpoint.predict_async(instances=[...]) my_predictions = response.predictions ``

  • Parameters

    • instances (List) – Required. The instances that are the input to the prediction call. A DeployedModel may have an upper limit on the number of instances it supports per request, and when it is exceeded the prediction call errors in case of AutoML Models, or, in case of customer created Models, the behaviour is as documented by that Model. The schema of any single instance may be specified via Endpoint’s DeployedModels’ [Model’s][google.cloud.aiplatform.v1beta1.DeployedModel.model] [PredictSchemata’s][google.cloud.aiplatform.v1beta1.Model.predict_schemata] instance_schema_uri.

    • parameters (Dict) – Optional. The parameters that govern the prediction. The schema of the parameters may be specified via Endpoint’s DeployedModels’ [Model’s ][google.cloud.aiplatform.v1beta1.DeployedModel.model] [PredictSchemata’s][google.cloud.aiplatform.v1beta1.Model.predict_schemata] parameters_schema_uri.

    • timeout (float) – Optional. The timeout for this request in seconds.

  • Returns

    Prediction with returned predictions and Model ID.

  • Return type

    prediction (aiplatform.Prediction)

property preview()

Return an Endpoint instance with preview features enabled.

raw_predict(body: bytes, headers: Dict[str, str])

Makes a prediction request using arbitrary headers.

Example usage:

my_endpoint = aiplatform.Endpoint(ENDPOINT_ID)
response = my_endpoint.raw_predict(

> body = b’{“instances”:[{“feat_1”:val_1, “feat_2”:val_2}]}’
> headers = {‘Content-Type’:’application/json’}

)
status_code = response.status_code
results = json.dumps(response.text)

  • Parameters

  • Returns

    A requests.models.Response object containing the status code and prediction results.

property resource_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Full qualified resource name.

to_dict()

Returns the resource proto as a dictionary.

property traffic_split(: Dict[str, int )

A map from a DeployedModel’s ID to the percentage of this Endpoint’s traffic that should be forwarded to that DeployedModel.

If a DeployedModel’s ID is not listed in this map, then it receives no traffic.

The traffic percentage values must add up to 100, or map must be empty if the Endpoint is to not accept any traffic at a moment.

undeploy(deployed_model_id: str, traffic_split: Optional[Dict[str, int]] = None, metadata: Optional[Sequence[Tuple[str, str]]] = (), sync=True)

Undeploys a deployed model.

The model to be undeployed should have no traffic or user must provide a new traffic_split with the remaining deployed models. Refer to Endpoint.traffic_split for the current traffic split mapping.

  • Parameters

    • deployed_model_id (str) – Required. The ID of the DeployedModel to be undeployed from the Endpoint.

    • traffic_split (Dict[str, *[int](https://python.readthedocs.io/en/latest/library/functions.html#int)]*) – Optional. A map of DeployedModel IDs to the percentage of this Endpoint’s traffic that should be forwarded to that DeployedModel. Required if undeploying a model with non-zero traffic from an Endpoint with multiple deployed models. The traffic percentage values must add up to 100, or map must be empty if the Endpoint is to not accept any traffic at the moment. If a DeployedModel’s ID is not listed in this map, then it receives no traffic.

    • metadata (Sequence[Tuple[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*]) – Optional. Strings which should be sent along with the request as metadata.

undeploy_all(sync: bool = True)

Undeploys every model deployed to this Endpoint.

  • Parameters

    sync (bool) – Whether to execute this method synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

update(display_name: Optional[str] = None, description: Optional[str] = None, labels: Optional[Dict[str, str]] = None, traffic_split: Optional[Dict[str, int]] = None, request_metadata: Optional[Sequence[Tuple[str, str]]] = (), update_request_timeout: Optional[float] = None)

Updates an endpoint.

Example usage:

my_endpoint = my_endpoint.update(

    display_name=’my-updated-endpoint’,
    description=’my updated description’,
    labels={‘key’: ‘value’},
    traffic_split={

    > ‘123456’: 20,
    > ‘234567’: 80,

    },

)

  • Parameters

    • display_name (str) – Optional. The display name of the Endpoint. The name can be up to 128 characters long and can be consist of any UTF-8 characters.

    • description (str) – Optional. The description of the Endpoint.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The labels with user-defined metadata to organize your Endpoints. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information and examples of labels.

    • traffic_split (Dict[str, *[int](https://python.readthedocs.io/en/latest/library/functions.html#int)]*) – Optional. A map from a DeployedModel’s ID to the percentage of this Endpoint’s traffic that should be forwarded to that DeployedModel. If a DeployedModel’s ID is not listed in this map, then it receives no traffic. The traffic percentage values must add up to 100, or map must be empty if the Endpoint is to not accept any traffic at a moment.

    • request_metadata (Sequence[Tuple[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*]) – Optional. Strings which should be sent along with the request as metadata.

    • update_request_timeout (float) – Optional. The timeout for the update request in seconds.

  • Returns

    Updated endpoint resource.

  • Return type

    Endpoint (aiplatform.Prediction)

  • Raises

    ValueError – If labels is not the correct format.

property update_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was last updated.

wait()

Helper method that blocks until all futures are complete.

class google.cloud.aiplatform.EntityType(entity_type_name: str, featurestore_id: Optional[str] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Bases: google.cloud.aiplatform.featurestore._entity_type._EntityType, google.cloud.aiplatform.base.PreviewMixin

Public managed EntityType resource for Vertex AI.

Retrieves an existing managed entityType given an entityType resource name or an entity_type ID.

Example Usage:

my_entity_type = aiplatform.EntityType(

entity_type_name=’projects/123/locations/us-central1/featurestores/my_featurestore_id/                entityTypes/my_entity_type_id’

) or my_entity_type = aiplatform.EntityType(

entity_type_name=’my_entity_type_id’, featurestore_id=’my_featurestore_id’,

)

  • Parameters

    • entity_type_name (str) – Required. A fully-qualified entityType resource name or an entity_type ID. Example: “projects/123/locations/us-central1/featurestores/my_featurestore_id/entityTypes/my_entity_type_id” or “my_entity_type_id” when project and location are initialized or passed, with featurestore_id passed.

    • featurestore_id (str) – Optional. Featurestore ID of an existing featurestore to retrieve entityType from, when entity_type_name is passed as entity_type ID.

    • project (str) – Optional. Project to retrieve entityType from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to retrieve entityType from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to retrieve this EntityType. Overrides credentials set in aiplatform.init.

batch_create_features(feature_configs: Dict[str, Dict[str, Union[bool, int, Dict[str, str], str]]], request_metadata: Optional[Sequence[Tuple[str, str]]] = (), sync: bool = True)

Batch creates Feature resources in this EntityType.

Example Usage:

my_entity_type = aiplatform.EntityType(

entity_type_name=’my_entity_type_id’,
featurestore_id=’my_featurestore_id’,

) my_entity_type.batch_create_features(

feature_configs={

“my_feature_id1”: {

    > “value_type”: “INT64”,

    },

“my_feature_id2”: {

    > “value_type”: “BOOL”,

    },

“my_feature_id3”: {

    > “value_type”: “STRING”,

    },

}

)

client_class()

alias of google.cloud.aiplatform.utils.FeaturestoreClientWithOverride

classmethod create(entity_type_id: str, featurestore_name: str, description: Optional[str] = None, labels: Optional[Dict[str, str]] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None, request_metadata: Optional[Sequence[Tuple[str, str]]] = (), sync: bool = True, create_request_timeout: Optional[float] = None)

Creates an EntityType resource in a Featurestore.

Example Usage:

my_entity_type = aiplatform.EntityType.create(

entity_type_id=’my_entity_type_id’,
featurestore_name=’projects/123/locations/us-central1/featurestores/my_featurestore_id’

) or my_entity_type = aiplatform.EntityType.create(

entity_type_id=’my_entity_type_id’, featurestore_name=’my_featurestore_id’,

)

  • Parameters

    • entity_type_id (str) – Required. The ID to use for the EntityType, which will become the final component of the EntityType’s resource name.

      This value may be up to 60 characters, and valid characters are [a-z0-9_]. The first character cannot be a number.

      The value must be unique within a featurestore.

    • featurestore_name (str) – Required. A fully-qualified featurestore resource name or a featurestore ID of an existing featurestore to create EntityType in. Example: “projects/123/locations/us-central1/featurestores/my_featurestore_id” or “my_featurestore_id” when project and location are initialized or passed.

    • description (str) – Optional. Description of the EntityType.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The labels with user-defined metadata to organize your EntityTypes. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information on and examples of labels. No more than 64 user labels can be associated with one EntityType (System labels are excluded).” System reserved label keys are prefixed with “aiplatform.googleapis.com/” and are immutable.

    • project (str) – Optional. Project to create EntityType in if featurestore_name is passed an featurestore ID. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to create EntityType in if featurestore_name is passed an featurestore ID. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to create EntityTypes. Overrides credentials set in aiplatform.init.

    • request_metadata (Sequence[Tuple[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*]) – Optional. Strings which should be sent along with the request as metadata.

    • sync (bool) – Optional. Whether to execute this creation synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

    • create_request_timeout (float) – Optional. The timeout for the create request in seconds.

  • Returns

    EntityType - entity_type resource object

create_feature(feature_id: str, value_type: str, description: Optional[str] = None, labels: Optional[Dict[str, str]] = None, request_metadata: Optional[Sequence[Tuple[str, str]]] = (), sync: bool = True, create_request_timeout: Optional[float] = None)

Creates a Feature resource in this EntityType.

Example Usage:

my_entity_type = aiplatform.EntityType(

entity_type_name=’my_entity_type_id’,
featurestore_id=’my_featurestore_id’,

) my_feature = my_entity_type.create_feature(

feature_id=’my_feature_id’, value_type=’INT64’,

)

  • Parameters

    • feature_id (str) – Required. The ID to use for the Feature, which will become the final component of the Feature’s resource name, which is immutable.

      This value may be up to 60 characters, and valid characters are [a-z0-9_]. The first character cannot be a number.

      The value must be unique within an EntityType.

    • value_type (str) – Required. Immutable. Type of Feature value. One of BOOL, BOOL_ARRAY, DOUBLE, DOUBLE_ARRAY, INT64, INT64_ARRAY, STRING, STRING_ARRAY, BYTES.

    • description (str) – Optional. Description of the Feature.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The labels with user-defined metadata to organize your Features. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information on and examples of labels. No more than 64 user labels can be associated with one Feature (System labels are excluded).” System reserved label keys are prefixed with “aiplatform.googleapis.com/” and are immutable.

    • request_metadata (Sequence[Tuple[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*]) – Optional. Strings which should be sent along with the request as metadata.

    • create_request_timeout (float) – Optional. The timeout for the create request in seconds.

    • sync (bool) – Optional. Whether to execute this creation synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

  • Returns

    featurestore.Feature - feature resource object

property create_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was created.

delete(sync: bool = True, force: bool = False)

Deletes this EntityType resource. If force is set to True, all features in this EntityType will be deleted prior to entityType deletion.

WARNING: This deletion is permanent.

  • Parameters

    • force (bool) – If set to true, any Features for this EntityType will also be deleted. (Otherwise, the request will only work if the EntityType has no Features.)

    • sync (bool) – Whether to execute this deletion synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

  • Raises

    FailedPrecondition – If features are created in this EntityType and force = False.

delete_features(feature_ids: List[str], sync: bool = True)

Deletes feature resources in this EntityType given their feature IDs. WARNING: This deletion is permanent.

  • Parameters

    • feature_ids (List[str]) – Required. The list of feature IDs to be deleted.

    • sync (bool) – Optional. Whether to execute this deletion synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

property display_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Display name of this resource.

property encryption_spec(: Optional[[google.cloud.aiplatform_v1.types.encryption_spec.EncryptionSpec](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.EncryptionSpec)_ )

Customer-managed encryption key options for this Vertex AI resource.

If this is set, then all resources created by this Vertex AI resource will be encrypted with the provided encryption key.

property featurestore_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Full qualified resource name of the managed featurestore in which this EntityType is.

property gca_resource(: [proto.message.Message](https://proto-plus-python.readthedocs.io/en/latest/reference/message.html#proto.message.Message )

The underlying resource proto representation.

get_feature(feature_id: str)

Retrieves an existing managed feature in this EntityType.

  • Parameters

    feature_id (str) – Required. The managed feature resource ID in this EntityType.

  • Returns

    featurestore.Feature - The managed feature resource object.

get_featurestore()

Retrieves the managed featurestore in which this EntityType is.

  • Returns

    featurestore.Featurestore - The managed featurestore in which this EntityType is.

ingest_from_bq(feature_ids: List[str], feature_time: Union[str, datetime.datetime], bq_source_uri: str, feature_source_fields: Optional[Dict[str, str]] = None, entity_id_field: Optional[str] = None, disable_online_serving: Optional[bool] = None, worker_count: Optional[int] = None, request_metadata: Optional[Sequence[Tuple[str, str]]] = (), sync: bool = True, ingest_request_timeout: Optional[float] = None)

Ingest feature values from BigQuery.

  • Parameters

    • feature_ids (List[str]) – Required. IDs of the Feature to import values of. The Features must exist in the target EntityType, or the request will fail.

    • feature_time (Union[str, *[datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime)]*) – Required. The feature_time can be one of:

        * The source column that holds the Feature

      timestamp for all Feature values in each entity.

      • A single Feature timestamp for all entities being imported. The timestamp must not have higher than millisecond precision.

    • bq_source_uri (str) – Required. BigQuery URI to the input table. .. rubric:: Example

      ’bq://project.dataset.table_name’

    • feature_source_fields (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. User defined dictionary to map ID of the Feature for importing values of to the source column for getting the Feature values from.

      Specify the features whose ID and source column are not the same. If not provided, the source column need to be the same as the Feature ID.

      Example

      feature_ids = [‘my_feature_id_1’, ‘my_feature_id_2’, ‘my_feature_id_3’]

      feature_source_fields = {

      ‘my_feature_id_1’: ‘my_feature_id_1_source_field’,

      }

      Note:

      The source column of ‘my_feature_id_1’ is ‘my_feature_id_1_source_field’, The source column of ‘my_feature_id_2’ is the ID of the feature, same for ‘my_feature_id_3’.

    • entity_id_field (str) – Optional. Source column that holds entity IDs. If not provided, entity IDs are extracted from the column named entity_id.

    • disable_online_serving (bool) – Optional. If set, data will not be imported for online serving. This is typically used for backfilling, where Feature generation timestamps are not in the timestamp range needed for online serving.

    • worker_count (int) – Optional. Specifies the number of workers that are used to write data to the Featurestore. Consider the online serving capacity that you require to achieve the desired import throughput without interfering with online serving. The value must be positive, and less than or equal to 100. If not set, defaults to using 1 worker. The low count ensures minimal impact on online serving performance.

    • request_metadata (Sequence[Tuple[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*]) – Optional. Strings which should be sent along with the request as metadata.

    • sync (bool) – Optional. Whether to execute this import synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

    • ingest_request_timeout (float) – Optional. The timeout for the ingest request in seconds.

  • Returns

    EntityType - The entityType resource object with feature values imported.

ingest_from_df(feature_ids: List[str], feature_time: Union[str, datetime.datetime], df_source: pd.DataFrame, feature_source_fields: Optional[Dict[str, str]] = None, entity_id_field: Optional[str] = None, request_metadata: Optional[Sequence[Tuple[str, str]]] = (), ingest_request_timeout: Optional[float] = None)

Ingest feature values from DataFrame.

NOTE: Calling this method will automatically create and delete a temporary bigquery dataset in the same GCP project, which will be used as the intermediary storage for ingesting feature values from dataframe to featurestore.

The call will return upon ingestion completes, where the feature values will be ingested into the entity_type.

  • Parameters

    • feature_ids (List[str]) – Required. IDs of the Feature to import values of. The Features must exist in the target EntityType, or the request will fail.

    • feature_time (Union[str, *[datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime)]*) – Required. The feature_time can be one of:

        * The source column that holds the Feature

      timestamp for all Feature values in each entity.

      Note:

        The dtype of the source column should be datetime64.


  * A single Feature timestamp for all entities

      being imported. The timestamp must not have higher than millisecond precision.

      Example:

        feature_time = datetime.datetime(year=2022, month=1, day=1, hour=11, minute=59, second=59)
  or
  feature_time_str = datetime.datetime.now().isoformat(sep=” “, timespec=”milliseconds”)
  feature_time = datetime.datetime.strptime(feature_time_str, “%Y-%m-%d %H:%M:%S.%f”)

    • df_source (pd.DataFrame) – Required. Pandas DataFrame containing the source data for ingestion.

    • feature_source_fields (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. User defined dictionary to map ID of the Feature for importing values of to the source column for getting the Feature values from.

      Specify the features whose ID and source column are not the same. If not provided, the source column need to be the same as the Feature ID.

      Example

      feature_ids = [‘my_feature_id_1’, ‘my_feature_id_2’, ‘my_feature_id_3’]

      feature_source_fields = {

      ‘my_feature_id_1’: ‘my_feature_id_1_source_field’,

      }

      Note:

      The source column of ‘my_feature_id_1’ is ‘my_feature_id_1_source_field’, The source column of ‘my_feature_id_2’ is the ID of the feature, same for ‘my_feature_id_3’.

    • entity_id_field (str) – Optional. Source column that holds entity IDs. If not provided, entity IDs are extracted from the column named entity_id.

    • request_metadata (Sequence[Tuple[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*]) – Optional. Strings which should be sent along with the request as metadata.

    • ingest_request_timeout (float) – Optional. The timeout for the ingest request in seconds.

  • Returns

    EntityType - The entityType resource object with feature values imported.

ingest_from_gcs(feature_ids: List[str], feature_time: Union[str, datetime.datetime], gcs_source_uris: Union[str, List[str]], gcs_source_type: str, feature_source_fields: Optional[Dict[str, str]] = None, entity_id_field: Optional[str] = None, disable_online_serving: Optional[bool] = None, worker_count: Optional[int] = None, request_metadata: Optional[Sequence[Tuple[str, str]]] = (), sync: bool = True, ingest_request_timeout: Optional[float] = None)

Ingest feature values from GCS.

  • Parameters

    • feature_ids (List[str]) – Required. IDs of the Feature to import values of. The Features must exist in the target EntityType, or the request will fail.

    • feature_time (Union[str, *[datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime)]*) – Required. The feature_time can be one of:

        * The source column that holds the Feature

      timestamp for all Feature values in each entity.

      • A single Feature timestamp for all entities being imported. The timestamp must not have higher than millisecond precision.

    • gcs_source_uris (Union[str, **List[str]]) – Required. Google Cloud Storage URI(-s) to the input file(s). May contain wildcards. For more information on wildcards, see https://cloud.google.com/storage/docs/gsutil/addlhelp/WildcardNames. .. rubric:: Example

      [“gs://my_bucket/my_file_1.csv”, “gs://my_bucket/my_file_2.csv”] or “gs://my_bucket/my_file.avro”

    • gcs_source_type (str) – Required. The type of the input file(s) provided by gcs_source_uris, the value of gcs_source_type can only be either csv, or avro.

    • feature_source_fields (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. User defined dictionary to map ID of the Feature for importing values of to the source column for getting the Feature values from.

      Specify the features whose ID and source column are not the same. If not provided, the source column need to be the same as the Feature ID.

      Example

      feature_ids = [‘my_feature_id_1’, ‘my_feature_id_2’, ‘my_feature_id_3’]

      feature_source_fields = {

      ‘my_feature_id_1’: ‘my_feature_id_1_source_field’,

      }

      Note:

      The source column of ‘my_feature_id_1’ is ‘my_feature_id_1_source_field’, The source column of ‘my_feature_id_2’ is the ID of the feature, same for ‘my_feature_id_3’.

    • entity_id_field (str) – Optional. Source column that holds entity IDs. If not provided, entity IDs are extracted from the column named entity_id.

    • disable_online_serving (bool) – Optional. If set, data will not be imported for online serving. This is typically used for backfilling, where Feature generation timestamps are not in the timestamp range needed for online serving.

    • worker_count (int) – Optional. Specifies the number of workers that are used to write data to the Featurestore. Consider the online serving capacity that you require to achieve the desired import throughput without interfering with online serving. The value must be positive, and less than or equal to 100. If not set, defaults to using 1 worker. The low count ensures minimal impact on online serving performance.

    • request_metadata (Sequence[Tuple[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*]) – Optional. Strings which should be sent along with the request as metadata.

    • sync (bool) – Optional. Whether to execute this import synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

    • ingest_request_timeout (float) – Optional. The timeout for the ingest request in seconds.

  • Returns

    EntityType - The entityType resource object with feature values imported.

  • Raises

    ValueError if gcs_source_type is not supported. –

property labels(: Dict[str, str )

User-defined labels containing metadata about this resource.

Read more about labels at https://goo.gl/xmQnxf

classmethod list(featurestore_name: str, filter: Optional[str] = None, order_by: Optional[str] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Lists existing managed entityType resources in a featurestore, given a featurestore resource name or a featurestore ID.

Example Usage:

my_entityTypes = aiplatform.EntityType.list(

featurestore_name=’projects/123/locations/us-central1/featurestores/my_featurestore_id’

) or my_entityTypes = aiplatform.EntityType.list(

featurestore_name=’my_featurestore_id’

)

  • Parameters

    • featurestore_name (str) – Required. A fully-qualified featurestore resource name or a featurestore ID of an existing featurestore to list entityTypes in. Example: “projects/123/locations/us-central1/featurestores/my_featurestore_id” or “my_featurestore_id” when project and location are initialized or passed.

    • filter (str) – Optional. Lists the EntityTypes that match the filter expression. The following filters are supported:

      • create_time: Supports =, !=, <, >, >=, and <= comparisons. Values must be in RFC 3339 format.

      • update_time: Supports =, !=, <, >, >=, and <= comparisons. Values must be in RFC 3339 format.

      • labels: Supports key-value equality as well as key presence.

      Examples:

      • create_time > "2020-01-31T15:30:00.000000Z" OR update_time > "2020-01-31T15:30:00.000000Z" –> EntityTypes created or updated after 2020-01-31T15:30:00.000000Z.

      • labels.active = yes AND labels.env = prod –> EntityTypes having both (active: yes) and (env: prod) labels.

      • labels.env: \* –> Any EntityType which has a label with ‘env’ as the key.

    • order_by (str) – Optional. A comma-separated list of fields to order by, sorted in ascending order. Use “desc” after a field name for descending.

      Supported fields:

      • entity_type_id

      • create_time

      • update_time

    • project (str) – Optional. Project to list entityTypes in. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to list entityTypes in. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to list entityTypes. Overrides credentials set in aiplatform.init.

  • Returns

    List[EntityType] - A list of managed entityType resource objects

list_features(filter: Optional[str] = None, order_by: Optional[str] = None)

Lists existing managed feature resources in this EntityType.

Example Usage:

my_entity_type = aiplatform.EntityType(

entity_type_name=’my_entity_type_id’,
featurestore_id=’my_featurestore_id’,

) my_entityType.list_features()

  • Parameters

    • filter (str) – Optional. Lists the Features that match the filter expression. The following filters are supported:

      • value_type: Supports = and != comparisons.

      • create_time: Supports =, !=, <, >, >=, and <= comparisons. Values must be in RFC 3339 format.

      • update_time: Supports =, !=, <, >, >=, and <= comparisons. Values must be in RFC 3339 format.

      • labels: Supports key-value equality as well as key presence.

      Examples:

      • value_type = DOUBLE –> Features whose type is DOUBLE.

      • create_time > "2020-01-31T15:30:00.000000Z" OR update_time > "2020-01-31T15:30:00.000000Z" –> EntityTypes created or updated after 2020-01-31T15:30:00.000000Z.

      • labels.active = yes AND labels.env = prod –> Features having both (active: yes) and (env: prod) labels.

      • labels.env: \* –> Any Feature which has a label with ‘env’ as the key.

    • order_by (str) – Optional. A comma-separated list of fields to order by, sorted in ascending order. Use “desc” after a field name for descending. Supported fields:

      • feature_id

      • value_type

      • create_time

      • update_time

  • Returns

    List[featurestore.Feature] - A list of managed feature resource objects.

property name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Name of this resource.

property preview(: google.cloud.aiplatform.base.PreviewClas )

Exposes features available in preview for this class.

read(entity_ids: Union[str, List[str]], feature_ids: Union[str, List[str]] = '*', request_metadata: Optional[Sequence[Tuple[str, str]]] = (), read_request_timeout: Optional[float] = None)

Reads feature values for given feature IDs of given entity IDs in this EntityType.

  • Parameters

    • entity_ids (Union[str, **List[str]]) – Required. ID for a specific entity, or a list of IDs of entities to read Feature values of. The maximum number of IDs is 100 if a list.

    • feature_ids (Union[str, **List[str]]) – Required. ID for a specific feature, or a list of IDs of Features in the EntityType for reading feature values. Default to “*”, where value of all features will be read.

    • request_metadata (Sequence[Tuple[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*]) – Optional. Strings which should be sent along with the request as metadata.

    • read_request_timeout (float) – Optional. The timeout for the read request in seconds.

  • Returns

    entities’ feature values in DataFrame

  • Return type

    pd.DataFrame

property resource_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Full qualified resource name.

to_dict()

Returns the resource proto as a dictionary.

update(description: Optional[str] = None, labels: Optional[Dict[str, str]] = None, request_metadata: Sequence[Tuple[str, str]] = (), update_request_timeout: Optional[float] = None)

Updates an existing managed entityType resource.

Example Usage:

my_entity_type = aiplatform.EntityType(

entity_type_name=’my_entity_type_id’,
featurestore_id=’my_featurestore_id’,

) my_entity_type.update(

description=’update my description’,

)

  • Parameters

    • description (str) – Optional. Description of the EntityType.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The labels with user-defined metadata to organize your EntityTypes. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information on and examples of labels. No more than 64 user labels can be associated with one Feature (System labels are excluded).” System reserved label keys are prefixed with “aiplatform.googleapis.com/” and are immutable.

    • request_metadata (Sequence[Tuple[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*]) – Required. Strings which should be sent along with the request as metadata.

    • update_request_timeout (float) – Optional. The timeout for the update request in seconds.

  • Returns

    EntityType - The updated entityType resource object.

property update_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was last updated.

wait()

Helper method that blocks until all futures are complete.

write_feature_values(instances: Union[List[google.cloud.aiplatform_v1.types.featurestore_online_service.WriteFeatureValuesPayload], Dict[str, Dict[str, Union[int, str, float, bool, bytes, List[int], List[str], List[float], List[bool]]]], pd.DataFrame], feature_time: Union[str, datetime.datetime] = None)

Streaming ingestion. Write feature values directly to Feature Store.

``

` my_entity_type = aiplatform.EntityType(

entity_type_name=”my_entity_type_id”, featurestore_id=”my_featurestore_id”,

)

writing feature values from a pandas DataFrame without feature timestamp column.

In this case, current timestamp will be applied to all data.

my_dataframe = pd.DataFrame(

data = [

{“entity_id”: “movie_01”, “average_rating”: 4.9}

], columns=[“entity_id”, “average_rating”],

) my_dataframe = my_df.set_index(“entity_id”)

my_entity_type.write_feature_values(

instances=my_df

)

writing feature values from a pandas DataFrame with feature timestamp column

Example of datetime creation.

feature_time = datetime.datetime(year=2022, month=1, day=1, hour=11, minute=59, second=59) or feature_time_str = datetime.datetime.now().isoformat(sep=” “, timespec=”milliseconds”) feature_time = datetime.datetime.strptime(feature_time_str, “%Y-%m-%d %H:%M:%S.%f”)

my_dataframe = pd.DataFrame(

data = [

    {“entity_id”: “movie_01”, “average_rating”: 4.9,
    “feature_timestamp”: feature_time}

],
columns=[“entity_id”, “average_rating”, “feature_timestamp”],

)

my_dataframe = my_df.set_index(“entity_id”) my_entity_type.write_feature_values(

instances=my_df, feature_time=”feature_timestamp”

)

writing feature values with a timestamp. The timestamp will be applied to the entire Dataframe.

my_dataframe = pd.DataFrame(

data = [

{“entity_id”: “movie_01”, “average_rating”: 4.9}

], columns=[“entity_id”, “average_rating”],

) my_dataframe = my_df.set_index(“entity_id”) my_entity_type.write_feature_values(

instances=my_df, feature_time=feature_time

)

writing feature values from a Python dict without timestamp column.

In this case, current timestamp will be applied to all data.

my_data_dict = {

“movie_02” : {“average_rating”: 3.7}

}

my_entity_type.write_feature_values(

instances=my_data_dict

)

writing feature values from a Python dict with timestamp column

my_data_dict = {

“movie_02” : {“average_rating”: 3.7, “feature_timestamp”: timestmap}}

}

my_entity_type.write_feature_values(

instances=my_data_dict, feature_time=”feature_timestamp”

)

writing feature values from a Python dict and apply the same Feature_Timestamp

my_data_dict = {

“movie_02” : {“average_rating”: 3.7}

}

my_entity_type.write_feature_values(

instances=my_data_dict, feature_time=feature_time

)

writing feature values from a list of WriteFeatureValuesPayload objects

payloads = [

gca_featurestore_online_service.WriteFeatureValuesPayload(

entity_id=”movie_03”,
feature_values={

> > “average_rating”: featurestore_online_service.FeatureValue(

> >     string_value=”test”,
> >     metadata=featurestore_online_service.FeatureValue.Metadata(

> >     > generate_time=timestmap

> >     )

> }

}

)

]

when instance is WriteFeatureValuesPayload,

feature_time param of write_feature_values() is ignored.

my_entity_type.write_feature_values(

instances=payloads

)

reading back written feature values

my_entity_type.read(

entity_ids=[“movie_01”, “movie_02”, “movie_03”]

  • Parameters

    • ( (instances) – Union[

      List[gca_featurestore_online_service.WriteFeatureValuesPayload], Dict[str, Dict[str, Union[int, str, float, bool, bytes,

      List[int], List[str], List[float], List[bool]]]],

      pd.Dataframe]):

      Required. Feature values to be written to the Feature Store that can take the form of a list of WriteFeatureValuesPayload objects, a Python dict of the form {entity_id : {feature_id : feature_value}, …}, or a pandas Dataframe, where the index column holds the unique entity ID strings and each remaining column represents a feature. Each row in the pandas Dataframe represents an entity, which has an entity ID and its associated feature values. Currently, a single payload can be written in a single request.

    • Union[str (feature_time) – Optional. Either column name in DataFrame or Dict which contains timestamp value, or datetime to apply to the entire DataFrame or Dict. Timestamp will be applied to generate_timestmap in all FeatureValue. If not provided, curreent timestamp is used. This param is not used when instances is List[WriteFeatureValuesPayload].

    • datetime.datetime] – Optional. Either column name in DataFrame or Dict which contains timestamp value, or datetime to apply to the entire DataFrame or Dict. Timestamp will be applied to generate_timestmap in all FeatureValue. If not provided, curreent timestamp is used. This param is not used when instances is List[WriteFeatureValuesPayload].

  • Returns

    EntityType - The updated EntityType object.

class google.cloud.aiplatform.Execution(execution_name: str, *, metadata_store_id: str = 'default', project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Bases: google.cloud.aiplatform.metadata.resource._Resource

Metadata Execution resource for Vertex AI

Retrieves an existing Metadata Execution given a resource name or ID.

  • Parameters

    • execution_name (str) – Required. A fully-qualified resource name or resource ID of the Execution. Example: “projects/123/locations/us-central1/metadataStores/default/executions/my-resource”. or “my-resource” when project and location are initialized or passed.

    • metadata_store_id (str) – Optional. MetadataStore to retrieve Execution from. If not set, metadata_store_id is set to “default”. If execution_name is a fully-qualified resource, its metadata_store_id overrides this one.

    • project (str) – Optional. Project to retrieve the artifact from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to retrieve the Execution from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to retrieve this Execution. Overrides credentials set in aiplatform.init.

assign_input_artifacts(artifacts: List[Union[google.cloud.aiplatform.metadata.artifact.Artifact, google.cloud.aiplatform.models.Model]])

Assigns Artifacts as inputs to this Executions.

  • Parameters

    artifacts (List[Union[artifact.Artifact, **models.Model]]) – Required. Artifacts to assign as input.

assign_output_artifacts(artifacts: List[Union[google.cloud.aiplatform.metadata.artifact.Artifact, google.cloud.aiplatform.models.Model]])

Assigns Artifacts as outputs to this Executions.

  • Parameters

    artifacts (List[Union[artifact.Artifact, **models.Model]]) – Required. Artifacts to assign as input.

client_class()

alias of google.cloud.aiplatform.utils.MetadataClientWithOverride

classmethod create(schema_title: str, *, state: google.cloud.aiplatform_v1.types.execution.Execution.State = State.RUNNING, resource_id: Optional[str] = None, display_name: Optional[str] = None, schema_version: Optional[str] = None, metadata: Optional[Dict[str, Any]] = None, description: Optional[str] = None, metadata_store_id: str = 'default', project: Optional[str] = None, location: Optional[str] = None, credentials=typing.Optional[google.auth.credentials.Credentials])

Creates a new Metadata Execution.

  • Parameters

    • schema_title (str) – Required. schema_title identifies the schema title used by the Execution.

    • state (gca_execution.Execution.State.RUNNING) – Optional. State of this Execution. Defaults to RUNNING.

    • resource_id (str) – Optional. The <resource_id> portion of the Execution name with the format. This is globally unique in a metadataStore: projects/123/locations/us-central1/metadataStores/<metadata_store_id>/executions/<resource_id>.

    • display_name (str) – Optional. The user-defined name of the Execution.

    • schema_version (str) – Optional. schema_version specifies the version used by the Execution. If not set, defaults to use the latest version.

    • metadata (Dict) – Optional. Contains the metadata information that will be stored in the Execution.

    • description (str) – Optional. Describes the purpose of the Execution to be created.

    • metadata_store_id (str) – Optional. The <metadata_store_id> portion of the resource name with the format: projects/123/locations/us-central1/metadataStores/<metadata_store_id>/artifacts/<resource_id> If not provided, the MetadataStore’s ID will be set to “default”.

    • project (str) – Optional. Project used to create this Execution. Overrides project set in aiplatform.init.

    • location (str) – Optional. Location used to create this Execution. Overrides location set in aiplatform.init.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials used to create this Execution. Overrides credentials set in aiplatform.init.

  • Returns

    Instantiated representation of the managed Metadata Execution.

  • Return type

    Execution

property create_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was created.

delete(sync: bool = True)

Deletes this Vertex AI resource. WARNING: This deletion is permanent.

  • Parameters

    sync (bool) – Whether to execute this deletion synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

property display_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Display name of this resource.

property encryption_spec(: Optional[[google.cloud.aiplatform_v1.types.encryption_spec.EncryptionSpec](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.EncryptionSpec)_ )

Customer-managed encryption key options for this Vertex AI resource.

If this is set, then all resources created by this Vertex AI resource will be encrypted with the provided encryption key.

property gca_resource(: [proto.message.Message](https://proto-plus-python.readthedocs.io/en/latest/reference/message.html#proto.message.Message )

The underlying resource proto representation.

classmethod get(resource_id: str, metadata_store_id: str = 'default', project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Retrieves a Metadata resource.

  • Parameters

    • resource_id (str) – Required. The <resource_id> portion of the resource name with the format: projects/123/locations/us-central1/metadataStores/<metadata_store_id>/<resource_noun>/<resource_id>.

    • metadata_store_id (str) – The <metadata_store_id> portion of the resource name with the format: projects/123/locations/us-central1/metadataStores/<metadata_store_id>/<resource_noun>/<resource_id> If not provided, the MetadataStore’s ID will be set to “default”.

    • project (str) – Project used to retrieve or create this resource. Overrides project set in aiplatform.init.

    • location (str) – Location used to retrieve or create this resource. Overrides location set in aiplatform.init.

    • credentials (auth_credentials.Credentials) – Custom credentials used to retrieve or create this resource. Overrides credentials set in aiplatform.init.

  • Returns

    Instantiated representation of the managed Metadata resource or None if no resource was found.

  • Return type

    resource (_Resource)

get_input_artifacts()

Get the input Artifacts of this Execution.

  • Returns

    List of input Artifacts.

classmethod get_or_create(resource_id: str, schema_title: str, display_name: Optional[str] = None, schema_version: Optional[str] = None, description: Optional[str] = None, metadata: Optional[Dict] = None, metadata_store_id: str = 'default', project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Retrieves or Creates (if it does not exist) a Metadata resource.

  • Parameters

    • resource_id (str) – Required. The <resource_id> portion of the resource name with the format: projects/123/locations/us-central1/metadataStores/<metadata_store_id>/<resource_noun>/<resource_id>.

    • schema_title (str) – Required. schema_title identifies the schema title used by the resource.

    • display_name (str) – Optional. The user-defined name of the resource.

    • schema_version (str) – Optional. schema_version specifies the version used by the resource. If not set, defaults to use the latest version.

    • description (str) – Optional. Describes the purpose of the resource to be created.

    • metadata (Dict) – Optional. Contains the metadata information that will be stored in the resource.

    • metadata_store_id (str) – The <metadata_store_id> portion of the resource name with the format: projects/123/locations/us-central1/metadataStores/<metadata_store_id>/<resource_noun>/<resource_id> If not provided, the MetadataStore’s ID will be set to “default”.

    • project (str) – Project used to retrieve or create this resource. Overrides project set in aiplatform.init.

    • location (str) – Location used to retrieve or create this resource. Overrides location set in aiplatform.init.

    • credentials (auth_credentials.Credentials) – Custom credentials used to retrieve or create this resource. Overrides credentials set in aiplatform.init.

  • Returns

    Instantiated representation of the managed Metadata resource.

  • Return type

    resource (_Resource)

get_output_artifacts()

Get the output Artifacts of this Execution.

  • Returns

    List of output Artifacts.

property labels(: Dict[str, str )

User-defined labels containing metadata about this resource.

Read more about labels at https://goo.gl/xmQnxf

classmethod list(filter: Optional[str] = None, metadata_store_id: str = 'default', project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None, order_by: Optional[str] = None)

List resources that match the list filter in target metadataStore.

  • Parameters

    • filter (str) – Optional. A query to filter available resources for matching results.

    • metadata_store_id (str) – The <metadata_store_id> portion of the resource name with the format: projects/123/locations/us-central1/metadataStores/<metadata_store_id>/<resource_noun>/<resource_id> If not provided, the MetadataStore’s ID will be set to “default”.

    • project (str) – Project used to create this resource. Overrides project set in aiplatform.init.

    • location (str) – Location used to create this resource. Overrides location set in aiplatform.init.

    • credentials (auth_credentials.Credentials) – Custom credentials used to create this resource. Overrides credentials set in aiplatform.init.

    • order_by (str) – Optional. How the list of messages is ordered. Specify the values to order by and an ordering operation. The default sorting order is ascending. To specify descending order for a field, users append a ” desc” suffix; for example: “foo desc, bar”. Subfields are specified with a . character, such as foo.bar. see https://google.aip.dev/132#ordering for more details.

  • Returns

    a list of managed Metadata resource.

  • Return type

    resources (sequence[_Resource])

property name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Name of this resource.

property resource_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Full qualified resource name.

property state(: [google.cloud.aiplatform_v1.types.execution.Execution.State](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.Execution.State_ )

State of this Execution.

sync_resource()

Syncs local resource with the resource in metadata store.

to_dict()

Returns the resource proto as a dictionary.

update(state: Optional[google.cloud.aiplatform_v1.types.execution.Execution.State] = None, description: Optional[str] = None, metadata: Optional[Dict[str, Any]] = None)

Update this Execution.

  • Parameters

    • state (gca_execution.Execution.State) – Optional. State of this Execution.

    • description (str) – Optional. Describes the purpose of the Execution to be created.

    • metadata (Dict[str, **Any) – Optional. Contains the metadata information that will be stored in the Execution.

property update_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was last updated.

wait()

Helper method that blocks until all futures are complete.

class google.cloud.aiplatform.Experiment(experiment_name: str, *, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Bases: object

Represents a Vertex AI Experiment resource.

\py my_experiment = aiplatform.Experiment('my-experiment') ``

  • Parameters

    • experiment_name (str) – Required. The name or resource name of this experiment.

      Resource name is of the format: projects/123/locations/us-central1/metadataStores/default/contexts/my-experiment

    • project (str) – Optional. Project where this experiment is located. Overrides project set in aiplatform.init.

    • location (str) – Optional. Location where this experiment is located. Overrides location set in aiplatform.init.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials used to retrieve this experiment. Overrides credentials set in aiplatform.init.

assign_backing_tensorboard(tensorboard: Union[google.cloud.aiplatform.tensorboard.tensorboard_resource.Tensorboard, str])

Assigns tensorboard as backing tensorboard to support time series metrics logging.

\py tb = aiplatform.Tensorboard('tensorboard-resource-id') my_experiment = aiplatform.Experiment('my-experiment') my_experiment.assign_backing_tensorboard(tb) ``

property backing_tensorboard_resource_name(: Optional[str )

The Tensorboard resource associated with this Experiment if there is one.

classmethod create(experiment_name: str, *, description: Optional[str] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Creates a new experiment in Vertex AI Experiments.

\py my_experiment = aiplatform.Experiment.create('my-experiment', description='my description') ``

  • Parameters

    • experiment_name (str) – Required. The name of this experiment.

    • description (str) – Optional. Describes this experiment’s purpose.

    • project (str) – Optional. Project where this experiment will be created. Overrides project set in aiplatform.init.

    • location (str) – Optional. Location where this experiment will be created. Overrides location set in aiplatform.init.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials used to create this experiment. Overrides credentials set in aiplatform.init.

  • Returns

    The newly created experiment.

property dashboard_url(: Optional[str )

Cloud console URL for this resource.

delete(*, delete_backing_tensorboard_runs: bool = False)

Deletes this experiment all the experiment runs under this experiment

Does not delete Pipeline runs, Artifacts, or Executions associated to this experiment or experiment runs in this experiment.

\py my_experiment = aiplatform.Experiment('my-experiment') my_experiment.delete(delete_backing_tensorboard_runs=True) ``

  • Parameters

    delete_backing_tensorboard_runs (bool) – Optional. If True will also delete the Tensorboard Runs associated to the experiment runs under this experiment that we used to store time series metrics.

classmethod get(experiment_name: str, *, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Gets experiment if one exists with this experiment_name in Vertex AI Experiments.

  • Parameters

    • experiment_name (str) – Required. The name of this experiment.

    • project (str) – Optional. Project used to retrieve this resource. Overrides project set in aiplatform.init.

    • location (str) – Optional. Location used to retrieve this resource. Overrides location set in aiplatform.init.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials used to retrieve this resource. Overrides credentials set in aiplatform.init.

  • Returns

    Vertex AI experiment or None if no resource was found.

get_backing_tensorboard_resource()

Get the backing tensorboard for this experiment if one exists.

\py my_experiment = aiplatform.Experiment('my-experiment') tb = my_experiment.get_backing_tensorboard_resource() ``

  • Returns

    Backing Tensorboard resource for this experiment if one exists.

get_data_frame()

Get parameters, metrics, and time series metrics of all runs in this experiment as Dataframe.

\py my_experiment = aiplatform.Experiment('my-experiment') df = my_experiment.get_data_frame() ``

  • Returns

    Pandas Dataframe of Experiment Runs.

  • Return type

    pd.DataFrame

  • Raises

    ImportError – If pandas is not installed.

classmethod get_or_create(experiment_name: str, *, description: Optional[str] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Gets experiment if one exists with this experiment_name in Vertex AI Experiments.

Otherwise creates this experiment.

\py my_experiment = aiplatform.Experiment.get_or_create('my-experiment', description='my description') ``

  • Parameters

    • experiment_name (str) – Required. The name of this experiment.

    • description (str) – Optional. Describes this experiment’s purpose.

    • project (str) – Optional. Project where this experiment will be retrieved from or created. Overrides project set in aiplatform.init.

    • location (str) – Optional. Location where this experiment will be retrieved from or created. Overrides location set in aiplatform.init.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials used to retrieve or create this experiment. Overrides credentials set in aiplatform.init.

  • Returns

    Vertex AI experiment.

classmethod list(*, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

List all Vertex AI Experiments in the given project.

\py my_experiments = aiplatform.Experiment.list() ``

  • Parameters

    • project (str) – Optional. Project to list these experiments from. Overrides project set in aiplatform.init.

    • location (str) – Optional. Location to list these experiments from. Overrides location set in aiplatform.init.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to list these experiments. Overrides credentials set in aiplatform.init.

  • Returns

    List of Vertex AI experiments.

property name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

The name of this experiment.

property resource_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

The Metadata context resource name of this experiment.

class google.cloud.aiplatform.ExperimentRun(run_name: str, experiment: Union[google.cloud.aiplatform.metadata.experiment_resources.Experiment, str], *, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Bases: google.cloud.aiplatform.metadata.experiment_resources._ExperimentLoggable

A Vertex AI Experiment run.

\py my_run = aiplatform.ExperimentRun('my-run', experiment='my-experiment') ``

  • Parameters

    • run_name (str) – Required. The name of this run.

    • experiment (Union[experiment_resources.Experiment, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Required. The name or instance of this experiment.

    • project (str) – Optional. Project where this experiment run is located. Overrides project set in aiplatform.init.

    • location (str) – Optional. Location where this experiment run is located. Overrides location set in aiplatform.init.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials used to retrieve this experiment run. Overrides credentials set in aiplatform.init.

classmethod _init_subclass_(*, experiment_loggable_schemas: Tuple[google.cloud.aiplatform.metadata.experiment_resources._ExperimentLoggableSchema], **kwargs)

Register the metadata_schema for the subclass so Experiment can use it to retrieve the associated types.

usage:

class PipelineJob(…, experiment_loggable_schemas=

(_ExperimentLoggableSchema(title=’system.PipelineRun’), )

  • Parameters

    experiment_loggable_schemas – Tuple of the schema_title and type pairs that represent this resource. Note that a single item in the tuple will be most common. Currently only experiment run has multiple representation for backwards compatibility. Almost all schemas should be Contexts and Execution is currently only supported for backwards compatibility of experiment runs.

assign_backing_tensorboard(tensorboard: Union[google.cloud.aiplatform.tensorboard.tensorboard_resource.Tensorboard, str])

Assigns tensorboard as backing tensorboard to support timeseries metrics logging for this run.

associate_execution(execution: google.cloud.aiplatform.metadata.execution.Execution)

Associate an execution to this experiment run.

  • Parameters

    execution (aiplatform.Execution) – Execution to associate to this run.

classmethod create(run_name: str, *, experiment: Optional[Union[google.cloud.aiplatform.metadata.experiment_resources.Experiment, str]] = None, tensorboard: Optional[Union[google.cloud.aiplatform.tensorboard.tensorboard_resource.Tensorboard, str]] = None, state: google.cloud.aiplatform_v1.types.execution.Execution.State = State.RUNNING, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Creates a new experiment run in Vertex AI Experiments.

\py my_run = aiplatform.ExperimentRun.create('my-run', experiment='my-experiment') ``

  • Parameters

    • run_name (str) – Required. The name of this run.

    • experiment (Union[aiplatform.Experiment, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The name or instance of the experiment to create this run under. If not provided, will default to the experiment set in aiplatform.init.

    • tensorboard (Union[aiplatform.Tensorboard, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The resource name or instance of Vertex Tensorboard to use as the backing Tensorboard for time series metric logging. If not provided, will default to the the backing tensorboard of parent experiment if set. Must be in same project and location as this experiment run.

    • state (aiplatform.gapic.Execution.State) – Optional. The state of this run. Defaults to RUNNING.

    • project (str) – Optional. Project where this experiment will be created. Overrides project set in aiplatform.init.

    • location (str) – Optional. Location where this experiment will be created. Overrides location set in aiplatform.init.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials used to create this experiment. Overrides credentials set in aiplatform.init.

  • Returns

    The newly created experiment run.

property credentials(: [google.auth.credentials.Credentials](https://googleapis.dev/python/google-auth/latest/reference/google.auth.credentials.html#google.auth.credentials.Credentials )

The credentials used to access this experiment run.

delete(*, delete_backing_tensorboard_run: bool = False)

Deletes this experiment run.

Does not delete the executions, artifacts, or resources logged to this run.

  • Parameters

    delete_backing_tensorboard_run (bool) – Optional. Whether to delete the backing tensorboard run that stores time series metrics for this run.

end_run(*, state: google.cloud.aiplatform_v1.types.execution.Execution.State = State.COMPLETE)

Ends this experiment run and sets state to COMPLETE.

  • Parameters

    state (aiplatform.gapic.Execution.State) – Optional. Override the state at the end of run. Defaults to COMPLETE.

classmethod get(run_name: str, *, experiment: Optional[Union[google.cloud.aiplatform.metadata.experiment_resources.Experiment, str]] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Gets experiment run if one exists with this run_name.

  • Parameters

    • run_name (str) – Required. The name of this run.

    • experiment (Union[experiment_resources.Experiment, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The name or instance of this experiment. If not set, use the default experiment in aiplatform.init

    • project (str) – Optional. Project where this experiment run is located. Overrides project set in aiplatform.init.

    • location (str) – Optional. Location where this experiment run is located. Overrides location set in aiplatform.init.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials used to retrieve this experiment run. Overrides credentials set in aiplatform.init.

  • Returns

    Vertex AI experimentRun or None if no resource was found.

get_artifacts()

Get the list of artifacts associated to this run.

  • Returns

    List of artifacts associated to this run.

get_classification_metrics()

Get all the classification metrics logged to this run.

``

`

py my_run = aiplatform.ExperimentRun(‘my-run’, experiment=’my-experiment’) metric = my_run.get_classification_metrics()[0] print(metric)

{

“id”: “e6c893a4-222e-4c60-a028-6a3b95dfc109”,
“display_name”: “my-classification-metrics”,
“labels”: [“cat”, “dog”],
“matrix”: [[9,1], [1,9]],
“fpr”: [0.1, 0.5, 0.9],
“tpr”: [0.1, 0.7, 0.9],
“thresholds”: [0.9, 0.5, 0.1]

}

``

`

  • Returns

    List of classification metrics logged to this experiment run.

get_executions()

Get the List of Executions associated to this run

  • Returns

    List of executions associated to this run.

get_experiment_models()

Get all ExperimentModel associated to this experiment run.

  • Returns

    List of ExperimentModel instances associated this run.

get_logged_custom_jobs()

Get all CustomJobs associated to this experiment run.

  • Returns

    List of CustomJobs associated this run.

get_logged_pipeline_jobs()

Get all PipelineJobs associated to this experiment run.

  • Returns

    List of PipelineJobs associated this run.

get_metrics()

Get the summary metrics logged to this run.

  • Returns

    Summary metrics logged to this experiment run.

get_params()

Get the parameters logged to this run.

  • Returns

    Parameters logged to this experiment run.

get_state()

The state of this run.

get_time_series_data_frame()

Returns all time series in this Run as a DataFrame.

  • Returns

    Time series metrics in this Run as a Dataframe.

  • Return type

    pd.DataFrame

classmethod list(*, experiment: Optional[Union[google.cloud.aiplatform.metadata.experiment_resources.Experiment, str]] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

List the experiment runs for a given aiplatform.Experiment.

\py my_runs = aiplatform.ExperimentRun.list(experiment='my-experiment') ``

  • Parameters

    • experiment (Union[aiplatform.Experiment, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The experiment name or instance to list the experiment run from. If not provided, will use the experiment set in aiplatform.init.

    • project (str) – Optional. Project where this experiment is located. Overrides project set in aiplatform.init.

    • location (str) – Optional. Location where this experiment is located. Overrides location set in aiplatform.init.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials used to retrieve this experiment. Overrides credentials set in aiplatform.init.

  • Returns

    List of experiment runs.

property location(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

The location that this experiment is located in.

log(*, pipeline_job: Optional[google.cloud.aiplatform.pipeline_jobs.PipelineJob] = None)

Log a Vertex Resource to this experiment run.

\py my_run = aiplatform.ExperimentRun('my-run', experiment='my-experiment') my_job = aiplatform.PipelineJob(...) my_job.submit() my_run.log(my_job) ``

  • Parameters

    pipeline_job (aiplatform.PipelineJob) – Optional. A Vertex PipelineJob.

log_classification_metrics(*, labels: Optional[List[str]] = None, matrix: Optional[List[List[int]]] = None, fpr: Optional[List[float]] = None, tpr: Optional[List[float]] = None, threshold: Optional[List[float]] = None, display_name: Optional[str] = None)

Create an artifact for classification metrics and log to ExperimentRun. Currently supports confusion matrix and ROC curve.

``

`

py my_run = aiplatform.ExperimentRun(‘my-run’, experiment=’my-experiment’) classification_metrics = my_run.log_classification_metrics(

display_name=’my-classification-metrics’, labels=[‘cat’, ‘dog’], matrix=[[9, 1], [1, 9]], fpr=[0.1, 0.5, 0.9], tpr=[0.1, 0.7, 0.9], threshold=[0.9, 0.5, 0.1],

  • Parameters

    • labels (List[str]) – Optional. List of label names for the confusion matrix. Must be set if ‘matrix’ is set.

    • matrix (List[List[int]) – Optional. Values for the confusion matrix. Must be set if ‘labels’ is set.

    • fpr (List[float]) – Optional. List of false positive rates for the ROC curve. Must be set if ‘tpr’ or ‘thresholds’ is set.

    • tpr (List[float]) – Optional. List of true positive rates for the ROC curve. Must be set if ‘fpr’ or ‘thresholds’ is set.

    • threshold (List[float]) – Optional. List of thresholds for the ROC curve. Must be set if ‘fpr’ or ‘tpr’ is set.

    • display_name (str) – Optional. The user-defined name for the classification metric artifact.

  • Returns

    ClassificationMetrics artifact.

  • Raises

    ValueError – if ‘labels’ and ‘matrix’ are not set together or if ‘labels’ and ‘matrix’ are not in the same length or if ‘fpr’ and ‘tpr’ and ‘threshold’ are not set together or if ‘fpr’ and ‘tpr’ and ‘threshold’ are not in the same length

log_metrics(metrics: Dict[str, Union[float, int, str]])

Log single or multiple Metrics with specified key and value pairs.

Metrics with the same key will be overwritten.

\py my_run = aiplatform.ExperimentRun('my-run', experiment='my-experiment') my_run.log_metrics({'accuracy': 0.9, 'recall': 0.8}) ``

log_model(model: Union[sklearn.base.BaseEstimator, xgb.Booster, tf.Module], artifact_id: Optional[str] = None, *, uri: Optional[str] = None, input_example: Union[list, dict, pd.DataFrame, np.ndarray] = None, display_name: Optional[str] = None, metadata_store_id: Optional[str] = 'default', project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Saves a ML model into a MLMD artifact and log it to this ExperimentRun.

Supported model frameworks: sklearn, xgboost, tensorflow.

Example usage:

model = LinearRegression()
model.fit(X, y)
aiplatform.init(

> project=”my-project”,
> location=”my-location”,
> staging_bucket=”gs://my-bucket”,
> experiment=”my-exp”

)
with aiplatform.start_run(“my-run”):

> aiplatform.log_model(model, “my-sklearn-model”)

  • Parameters

    • model (Union["sklearn.base.BaseEstimator", **"xgb.Booster", **"tf.Module"]) – Required. A machine learning model.

    • artifact_id (str) – Optional. The resource id of the artifact. This id must be globally unique in a metadataStore. It may be up to 63 characters, and valid characters are [a-z0-9_-]. The first character cannot be a number or hyphen.

    • uri (str) – Optional. A gcs directory to save the model file. If not provided, gs://default-bucket/timestamp-uuid-frameworkName-model will be used. If default staging bucket is not set, a new bucket will be created.

    • input_example (Union[list, *[dict](https://python.readthedocs.io/en/latest/library/stdtypes.html#dict), pd.DataFrame, np.ndarray]*) – Optional. An example of a valid model input. Will be stored as a yaml file in the gcs uri. Accepts list, dict, pd.DataFrame, and np.ndarray The value inside a list must be a scalar or list. The value inside a dict must be a scalar, list, or np.ndarray.

    • display_name (str) – Optional. The display name of the artifact.

    • metadata_store_id (str) – Optional. The <metadata_store_id> portion of the resource name with the format: projects/123/locations/us-central1/metadataStores/<metadata_store_id>/artifacts/<resource_id> If not provided, the MetadataStore’s ID will be set to “default”.

    • project (str) – Optional. Project used to create this Artifact. Overrides project set in aiplatform.init.

    • location (str) – Optional. Location used to create this Artifact. Overrides location set in aiplatform.init.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials used to create this Artifact. Overrides credentials set in aiplatform.init.

  • Returns

    An ExperimentModel instance.

  • Raises

    ValueError – if model type is not supported.

log_params(params: Dict[str, Union[float, int, str]])

Log single or multiple parameters with specified key value pairs.

Parameters with the same key will be overwritten.

\py my_run = aiplatform.ExperimentRun('my-run', experiment='my-experiment') my_run.log_params({'learning_rate': 0.1, 'dropout_rate': 0.2}) ``

log_time_series_metrics(metrics: Dict[str, float], step: Optional[int] = None, wall_time: Optional[google.protobuf.timestamp_pb2.Timestamp] = None)

Logs time series metrics to backing TensorboardRun of this Experiment Run.

\py run.log_time_series_metrics({'accuracy': 0.9}, step=10) ``

  • Parameters

    • metrics (Dict[str, **Union[str, *[float](https://python.readthedocs.io/en/latest/library/functions.html#float)]*]) – Required. Dictionary of where keys are metric names and values are metric values.

    • step (int) – Optional. Step index of this data point within the run.

      If not provided, the latest step amongst all time series metrics already logged will be used.

    • wall_time (timestamp_pb2.Timestamp) – Optional. Wall clock timestamp when this data point is generated by the end user.

      If not provided, this will be generated based on the value from time.time()

  • Raises

    RuntimeError – If current experiment run doesn’t have a backing Tensorboard resource.

property name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

This run’s name used to identify this run within it’s Experiment.

property project(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

The project that this experiment run is located in.

property resource_id(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

The resource ID of this experiment run’s Metadata context.

The resource ID is the final part of the resource name: projects/{project}/locations/{location}/metadataStores/{metadatastore}/contexts/{resource ID}

property resource_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

This run’s Metadata context resource name.

In the format: projects/{project}/locations/{location}/metadataStores/{metadatastore}/contexts/{context}

property state(: [google.cloud.aiplatform_v1.types.execution.Execution.State](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.Execution.State_ )

The state of this run.

update_state(state: google.cloud.aiplatform_v1.types.execution.Execution.State)

Update the state of this experiment run.

\py my_run = aiplatform.ExperimentRun('my-run', experiment='my-experiment') my_run.update_state(state=aiplatform.gapic.Execution.State.COMPLETE) ``

  • Parameters

    state (aiplatform.gapic.Execution.State) – State of this run.

class google.cloud.aiplatform.Feature(feature_name: str, featurestore_id: Optional[str] = None, entity_type_id: Optional[str] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Bases: google.cloud.aiplatform.base.VertexAiResourceNounWithFutureManager

Managed feature resource for Vertex AI.

Retrieves an existing managed feature given a feature resource name or a feature ID.

Example Usage:

my_feature = aiplatform.Feature(

feature_name=’projects/123/locations/us-central1/featurestores/my_featurestore_id/                entityTypes/my_entity_type_id/features/my_feature_id’

) or my_feature = aiplatform.Feature(

feature_name=’my_feature_id’, featurestore_id=’my_featurestore_id’, entity_type_id=’my_entity_type_id’,

)

  • Parameters

    • feature_name (str) – Required. A fully-qualified feature resource name or a feature ID. Example: “projects/123/locations/us-central1/featurestores/my_featurestore_id/entityTypes/my_entity_type_id/features/my_feature_id” or “my_feature_id” when project and location are initialized or passed, with featurestore_id and entity_type_id passed.

    • featurestore_id (str) – Optional. Featurestore ID of an existing featurestore to retrieve feature from, when feature_name is passed as Feature ID.

    • entity_type_id (str) – Optional. EntityType ID of an existing entityType to retrieve feature from, when feature_name is passed as Feature ID. The EntityType must exist in the Featurestore if provided by the featurestore_id.

    • project (str) – Optional. Project to retrieve feature from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to retrieve feature from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to retrieve this Feature. Overrides credentials set in aiplatform.init.

  • Raises

    ValueError – If only one of featurestore_id or entity_type_id is provided.

client_class()

alias of google.cloud.aiplatform.utils.FeaturestoreClientWithOverride

classmethod create(feature_id: str, value_type: str, entity_type_name: str, featurestore_id: Optional[str] = None, description: Optional[str] = None, labels: Optional[Dict[str, str]] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None, request_metadata: Optional[Sequence[Tuple[str, str]]] = (), sync: bool = True, create_request_timeout: Optional[float] = None)

Creates a Feature resource in an EntityType.

Example Usage:

my_feature = aiplatform.Feature.create(

feature_id=’my_feature_id’,
value_type=’INT64’,
entity_type_name=’projects/123/locations/us-central1/featurestores/my_featurestore_id/                entityTypes/my_entity_type_id’

) or my_feature = aiplatform.Feature.create(

feature_id=’my_feature_id’, value_type=’INT64’, entity_type_name=’my_entity_type_id’, featurestore_id=’my_featurestore_id’,

)

  • Parameters

    • feature_id (str) – Required. The ID to use for the Feature, which will become the final component of the Feature’s resource name, which is immutable.

      This value may be up to 60 characters, and valid characters are [a-z0-9_]. The first character cannot be a number.

      The value must be unique within an EntityType.

    • value_type (str) – Required. Immutable. Type of Feature value. One of BOOL, BOOL_ARRAY, DOUBLE, DOUBLE_ARRAY, INT64, INT64_ARRAY, STRING, STRING_ARRAY, BYTES.

    • entity_type_name (str) – Required. A fully-qualified entityType resource name or an entity_type ID of an existing entityType to create Feature in. The EntityType must exist in the Featurestore if provided by the featurestore_id. Example: “projects/123/locations/us-central1/featurestores/my_featurestore_id/entityTypes/my_entity_type_id” or “my_entity_type_id” when project and location are initialized or passed, with featurestore_id passed.

    • featurestore_id (str) – Optional. Featurestore ID of an existing featurestore to create Feature in if entity_type_name is passed an entity_type ID.

    • description (str) – Optional. Description of the Feature.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The labels with user-defined metadata to organize your Features. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information on and examples of labels. No more than 64 user labels can be associated with one Feature (System labels are excluded).” System reserved label keys are prefixed with “aiplatform.googleapis.com/” and are immutable.

    • project (str) – Optional. Project to create Feature in if entity_type_name is passed an entity_type ID. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to create Feature in if entity_type_name is passed an entity_type ID. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to create Features. Overrides credentials set in aiplatform.init.

    • request_metadata (Sequence[Tuple[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*]) – Optional. Strings which should be sent along with the request as metadata.

    • sync (bool) – Optional. Whether to execute this creation synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

    • create_request_timeout (float) – Optional. The timeout for the create request in seconds.

  • Returns

    Feature - feature resource object

property create_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was created.

delete(sync: bool = True)

Deletes this Vertex AI resource. WARNING: This deletion is permanent.

  • Parameters

    sync (bool) – Whether to execute this deletion synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

property display_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Display name of this resource.

property encryption_spec(: Optional[[google.cloud.aiplatform_v1.types.encryption_spec.EncryptionSpec](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.EncryptionSpec)_ )

Customer-managed encryption key options for this Vertex AI resource.

If this is set, then all resources created by this Vertex AI resource will be encrypted with the provided encryption key.

property entity_type_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Full qualified resource name of the managed entityType in which this Feature is.

property featurestore_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Full qualified resource name of the managed featurestore in which this Feature is.

property gca_resource(: [proto.message.Message](https://proto-plus-python.readthedocs.io/en/latest/reference/message.html#proto.message.Message )

The underlying resource proto representation.

get_entity_type()

Retrieves the managed entityType in which this Feature is.

  • Returns

    featurestore.EntityType - The managed entityType in which this Feature is.

get_featurestore()

Retrieves the managed featurestore in which this Feature is.

  • Returns

    featurestore.Featurestore - The managed featurestore in which this Feature is.

property labels(: Dict[str, str )

User-defined labels containing metadata about this resource.

Read more about labels at https://goo.gl/xmQnxf

classmethod list(entity_type_name: str, featurestore_id: Optional[str] = None, filter: Optional[str] = None, order_by: Optional[str] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Lists existing managed feature resources in an entityType, given an entityType resource name or an entity_type ID.

Example Usage:

my_features = aiplatform.Feature.list(

entity_type_name=’projects/123/locations/us-central1/featurestores/my_featurestore_id/                entityTypes/my_entity_type_id’

) or my_features = aiplatform.Feature.list(

entity_type_name=’my_entity_type_id’, featurestore_id=’my_featurestore_id’,

)

  • Parameters

    • entity_type_name (str) – Required. A fully-qualified entityType resource name or an entity_type ID of an existing entityType to list features in. The EntityType must exist in the Featurestore if provided by the featurestore_id. Example: “projects/123/locations/us-central1/featurestores/my_featurestore_id/entityTypes/my_entity_type_id” or “my_entity_type_id” when project and location are initialized or passed, with featurestore_id passed.

    • featurestore_id (str) – Optional. Featurestore ID of an existing featurestore to list features in, when entity_type_name is passed as entity_type ID.

    • filter (str) – Optional. Lists the Features that match the filter expression. The following filters are supported:

      • value_type: Supports = and != comparisons.

      • create_time: Supports =, !=, <, >, >=, and <= comparisons. Values must be in RFC 3339 format.

      • update_time: Supports =, !=, <, >, >=, and <= comparisons. Values must be in RFC 3339 format.

      • labels: Supports key-value equality as well as key presence.

      Examples:

      • value_type = DOUBLE –> Features whose type is DOUBLE.

      • create_time > "2020-01-31T15:30:00.000000Z" OR update_time > "2020-01-31T15:30:00.000000Z" –> EntityTypes created or updated after 2020-01-31T15:30:00.000000Z.

      • labels.active = yes AND labels.env = prod –> Features having both (active: yes) and (env: prod) labels.

      • labels.env: \* –> Any Feature which has a label with ‘env’ as the key.

    • order_by (str) – Optional. A comma-separated list of fields to order by, sorted in ascending order. Use “desc” after a field name for descending. Supported fields:

      • feature_id

      • value_type

      • create_time

      • update_time

    • project (str) – Optional. Project to list features in. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to list features in. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to list features. Overrides credentials set in aiplatform.init.

  • Returns

    List[Feature] - A list of managed feature resource objects

property name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Name of this resource.

property resource_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Full qualified resource name.

classmethod search(query: Optional[str] = None, page_size: Optional[int] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Searches existing managed Feature resources.

Example Usage:

my_features = aiplatform.Feature.search()

  • Parameters

    • query (str) – Optional. Query string that is a conjunction of field-restricted queries and/or field-restricted filters. Field-restricted queries and filters can be combined using AND to form a conjunction.

      A field query is in the form FIELD:QUERY. This implicitly checks if QUERY exists as a substring within Feature’s FIELD. The QUERY and the FIELD are converted to a sequence of words (i.e. tokens) for comparison. This is done by:

      • Removing leading/trailing whitespace and tokenizing the search value. Characters that are not one of alphanumeric [a-zA-Z0-9], underscore _, or asterisk \* are treated as delimiters for tokens. \* is treated as a wildcard that matches characters within a token.

      • Ignoring case.

      • Prepending an asterisk to the first and appending an asterisk to the last token in QUERY.

      A QUERY must be either a singular token or a phrase. A phrase is one or multiple words enclosed in double quotation marks (“). With phrases, the order of the words is important. Words in the phrase must be matching in order and consecutively.

      Supported FIELDs for field-restricted queries:

      • feature_id

      • description

      • entity_type_id

      Examples:

      • feature_id: foo –> Matches a Feature with ID containing the substring foo (eg. foo, foofeature, barfoo).

      • feature_id: foo\*feature –> Matches a Feature with ID containing the substring foo\*feature (eg. foobarfeature).

      • feature_id: foo AND description: bar –> Matches a Feature with ID containing the substring foo and description containing the substring bar.

      Besides field queries, the following exact-match filters are supported. The exact-match filters do not support wildcards. Unlike field-restricted queries, exact-match filters are case-sensitive.

      • feature_id: Supports = comparisons.

      • description: Supports = comparisons. Multi-token filters should be enclosed in quotes.

      • entity_type_id: Supports = comparisons.

      • value_type: Supports = and != comparisons.

      • labels: Supports key-value equality as well as key presence.

      • featurestore_id: Supports = comparisons.

      Examples:

      • description = "foo bar" –> Any Feature with description exactly equal to foo bar

      • value_type = DOUBLE –> Features whose type is DOUBLE.

      • labels.active = yes AND labels.env = prod –> Features having both (active: yes) and (env: prod) labels.

      • labels.env: \* –> Any Feature which has a label with env as the key.

      This corresponds to the query field on the request instance; if request is provided, this should not be set.

    • page_size (int) – Optional. The maximum number of Features to return. The service may return fewer than this value. If unspecified, at most 100 Features will be returned. The maximum value is 100; any value greater than 100 will be coerced to 100.

    • project (str) – Optional. Project to list features in. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to list features in. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to list features. Overrides credentials set in aiplatform.init.

  • Returns

    List[Feature] - A list of managed feature resource objects

to_dict()

Returns the resource proto as a dictionary.

update(description: Optional[str] = None, labels: Optional[Dict[str, str]] = None, request_metadata: Optional[Sequence[Tuple[str, str]]] = (), update_request_timeout: Optional[float] = None)

Updates an existing managed feature resource.

Example Usage:

my_feature = aiplatform.Feature(

feature_name=’my_feature_id’,
featurestore_id=’my_featurestore_id’,
entity_type_id=’my_entity_type_id’,

) my_feature.update(

description=’update my description’,

)

  • Parameters

    • description (str) – Optional. Description of the Feature.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The labels with user-defined metadata to organize your Features. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information on and examples of labels. No more than 64 user labels can be associated with one Feature (System labels are excluded).” System reserved label keys are prefixed with “aiplatform.googleapis.com/” and are immutable.

    • request_metadata (Sequence[Tuple[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*]) – Optional. Strings which should be sent along with the request as metadata.

    • update_request_timeout (float) – Optional. The timeout for the update request in seconds.

  • Returns

    Feature - The updated feature resource object.

property update_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was last updated.

wait()

Helper method that blocks until all futures are complete.

class google.cloud.aiplatform.Featurestore(featurestore_name: str, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Bases: google.cloud.aiplatform.base.VertexAiResourceNounWithFutureManager

Managed featurestore resource for Vertex AI.

Retrieves an existing managed featurestore given a featurestore resource name or a featurestore ID.

Example Usage:

my_featurestore = aiplatform.Featurestore(

featurestore_name=’projects/123/locations/us-central1/featurestores/my_featurestore_id’

) or my_featurestore = aiplatform.Featurestore(

featurestore_name=’my_featurestore_id’

)

  • Parameters

    • featurestore_name (str) – Required. A fully-qualified featurestore resource name or a featurestore ID. Example: “projects/123/locations/us-central1/featurestores/my_featurestore_id” or “my_featurestore_id” when project and location are initialized or passed.

    • project (str) – Optional. Project to retrieve featurestore from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to retrieve featurestore from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to retrieve this Featurestore. Overrides credentials set in aiplatform.init.

batch_serve_to_bq(bq_destination_output_uri: str, serving_feature_ids: Dict[str, List[str]], read_instances_uri: str, pass_through_fields: Optional[List[str]] = None, feature_destination_fields: Optional[Dict[str, str]] = None, start_time: Optional[google.protobuf.timestamp_pb2.Timestamp] = None, request_metadata: Optional[Sequence[Tuple[str, str]]] = (), serve_request_timeout: Optional[float] = None, sync: bool = True)

Batch serves feature values to BigQuery destination

  • Parameters

    • bq_destination_output_uri (str) – Required. BigQuery URI to the detination table.

      Example

      ’bq://project.dataset.table_name’

      It requires an existing BigQuery destination Dataset, under the same project as the Featurestore.

    • serving_feature_ids (Dict[str, **List[str]]) – Required. A user defined dictionary to define the entity_types and their features for batch serve/read. The keys of the dictionary are the serving entity_type ids and the values are lists of serving feature ids in each entity_type.

      Example

      serving_feature_ids = {

      ‘my_entity_type_id_1’: [‘feature_id_1_1’, ‘feature_id_1_2’], ‘my_entity_type_id_2’: [‘feature_id_2_1’, ‘feature_id_2_2’],

      }

    • read_instances_uri (str) – Required. Read_instances_uri can be either BigQuery URI to an input table, or Google Cloud Storage URI to a csv file.

      Example

      ’bq://project.dataset.table_name’ or “gs://my_bucket/my_file.csv”

      Each read instance should consist of exactly one read timestamp and one or more entity IDs identifying entities of the corresponding EntityTypes whose Features are requested.

      Each output instance contains Feature values of requested entities concatenated together as of the read time.

      An example read instance may be foo_entity_id, bar_entity_id, 2020-01-01T10:00:00.123Z.

      An example output instance may be foo_entity_id, bar_entity_id, 2020-01-01T10:00:00.123Z, foo_entity_feature1_value, bar_entity_feature2_value.

      Timestamp in each read instance must be millisecond-aligned.

      The columns can be in any order.

      Values in the timestamp column must use the RFC 3339 format, e.g. 2012-07-30T10:43:17.123Z.

    • pass_through_fields (List[str]) – Optional. When not empty, the specified fields in the read_instances source will be joined as-is in the output, in addition to those fields from the Featurestore Entity.

      For BigQuery source, the type of the pass-through values will be automatically inferred. For CSV source, the pass-through values will be passed as opaque bytes.

    • feature_destination_fields (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. A user defined dictionary to map a feature’s fully qualified resource name to its destination field name. If the destination field name is not defined, the feature ID will be used as its destination field name.

      Example

      feature_destination_fields = {

      ‘projects/123/locations/us-central1/featurestores/fs_id/entityTypes/et_id1/features/f_id11’: ‘foo’, ‘projects/123/locations/us-central1/featurestores/fs_id/entityTypes/et_id2/features/f_id22’: ‘bar’,

      }

    • start_time (timestamp_pb2.Timestamp) – Optional. Excludes Feature values with feature generation timestamp before this timestamp. If not set, retrieve oldest values kept in Feature Store. Timestamp, if present, must not have higher than millisecond precision.

    • serve_request_timeout (float) – Optional. The timeout for the serve request in seconds.

  • Returns

    The featurestore resource object batch read feature values from.

  • Return type

    Featurestore

  • Raises

    • NotFound – if the BigQuery destination Dataset does not exist.

    • FailedPrecondition – if the BigQuery destination Dataset/Table is in a different project.

batch_serve_to_df(serving_feature_ids: Dict[str, List[str]], read_instances_df: pd.DataFrame, pass_through_fields: Optional[List[str]] = None, feature_destination_fields: Optional[Dict[str, str]] = None, start_time: Optional[google.protobuf.timestamp_pb2.Timestamp] = None, request_metadata: Optional[Sequence[Tuple[str, str]]] = (), serve_request_timeout: Optional[float] = None, bq_dataset_id: Optional[str] = None)

Batch serves feature values to pandas DataFrame

NOTE: Calling this method will automatically create and delete a temporary bigquery dataset in the same GCP project, which will be used as the intermediary storage for batch serve feature values from featurestore to dataframe.

  • Parameters

    • serving_feature_ids (Dict[str, **List[str]]) – Required. A user defined dictionary to define the entity_types and their features for batch serve/read. The keys of the dictionary are the serving entity_type ids and the values are lists of serving feature ids in each entity_type.

      Example

      serving_feature_ids = {

      ‘my_entity_type_id_1’: [‘feature_id_1_1’, ‘feature_id_1_2’], ‘my_entity_type_id_2’: [‘feature_id_2_1’, ‘feature_id_2_2’],

      }

    • read_instances_df (pd.DataFrame) – Required. Read_instances_df is a pandas DataFrame containing the read instances.

      Each read instance should consist of exactly one read timestamp and one or more entity IDs identifying entities of the corresponding EntityTypes whose Features are requested.

      Each output instance contains Feature values of requested entities concatenated together as of the read time.

      An example read_instances_df may be

      pd.DataFrame(

        data=[

      {

          “my_entity_type_id_1”: “my_entity_type_id_1_entity_1”,
          “my_entity_type_id_2”: “my_entity_type_id_2_entity_1”,
          “timestamp”: “2020-01-01T10:00:00.123Z”

  ],

      )

      An example batch_serve_output_df may be

      pd.DataFrame(

        data=[

      {

          “my_entity_type_id_1”: “my_entity_type_id_1_entity_1”,
          “my_entity_type_id_2”: “my_entity_type_id_2_entity_1”,
          “foo”: “feature_id_1_1_feature_value”,
          “feature_id_1_2”: “feature_id_1_2_feature_value”,
          “feature_id_2_1”: “feature_id_2_1_feature_value”,
          “bar”: “feature_id_2_2_feature_value”,
          “timestamp”: “2020-01-01T10:00:00.123Z”

  ],

      )

      Timestamp in each read instance must be millisecond-aligned.

      The columns can be in any order.

      Values in the timestamp column must use the RFC 3339 format, e.g. 2012-07-30T10:43:17.123Z.

    • pass_through_fields (List[str]) – Optional. When not empty, the specified fields in the read_instances source will be joined as-is in the output, in addition to those fields from the Featurestore Entity.

      For BigQuery source, the type of the pass-through values will be automatically inferred. For CSV source, the pass-through values will be passed as opaque bytes.

    • feature_destination_fields (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. A user defined dictionary to map a feature’s fully qualified resource name to its destination field name. If the destination field name is not defined, the feature ID will be used as its destination field name.

      Example

      feature_destination_fields = {

      ‘projects/123/locations/us-central1/featurestores/fs_id/entityTypes/et_id1/features/f_id11’: ‘foo’, ‘projects/123/locations/us-central1/featurestores/fs_id/entityTypes/et_id2/features/f_id22’: ‘bar’,

      }

    • serve_request_timeout (float) – Optional. The timeout for the serve request in seconds.

    • bq_dataset_id (str) – Optional. The full dataset ID for the BigQuery dataset to use for temporarily staging data. If specified, caller must have bigquery.tables.create permissions for Dataset.

    • start_time (timestamp_pb2.Timestamp) – Optional. Excludes Feature values with feature generation timestamp before this timestamp. If not set, retrieve oldest values kept in Feature Store. Timestamp, if present, must not have higher than millisecond precision.

  • Returns

    The pandas DataFrame containing feature values from batch serving.

  • Return type

    pd.DataFrame

batch_serve_to_gcs(gcs_destination_output_uri_prefix: str, gcs_destination_type: str, serving_feature_ids: Dict[str, List[str]], read_instances_uri: str, pass_through_fields: Optional[List[str]] = None, feature_destination_fields: Optional[Dict[str, str]] = None, start_time: Optional[google.protobuf.timestamp_pb2.Timestamp] = None, request_metadata: Optional[Sequence[Tuple[str, str]]] = (), sync: bool = True, serve_request_timeout: Optional[float] = None)

Batch serves feature values to GCS destination

  • Parameters

    • gcs_destination_output_uri_prefix (str) – Required. Google Cloud Storage URI to output directory. If the uri doesn’t end with ‘/’, a ‘/’ will be automatically appended. The directory is created if it doesn’t exist.

      Example

      ”gs://bucket/path/to/prefix”

    • gcs_destination_type (str) – Required. The type of the destination files(s), the value of gcs_destination_type can only be either csv, or tfrecord.

      For CSV format. Array Feature value types are not allowed in CSV format.

      For TFRecord format.

      Below are the mapping from Feature value type in Featurestore to Feature value type in TFRecord:

      Value type in Featurestore                 | Value type in TFRecord
DOUBLE, DOUBLE_ARRAY                       | FLOAT_LIST
INT64, INT64_ARRAY                         | INT64_LIST
STRING, STRING_ARRAY, BYTES                | BYTES_LIST
true -> byte_string("true"), false -> byte_string("false")
BOOL, BOOL_ARRAY (true, false)             | BYTES_LIST

    • serving_feature_ids (Dict[str, **List[str]]) – Required. A user defined dictionary to define the entity_types and their features for batch serve/read. The keys of the dictionary are the serving entity_type ids and the values are lists of serving feature ids in each entity_type.

      Example

      serving_feature_ids = {

      ‘my_entity_type_id_1’: [‘feature_id_1_1’, ‘feature_id_1_2’], ‘my_entity_type_id_2’: [‘feature_id_2_1’, ‘feature_id_2_2’],

      }

    • read_instances_uri (str) – Required. Read_instances_uri can be either BigQuery URI to an input table, or Google Cloud Storage URI to a csv file.

      Example

      ’bq://project.dataset.table_name’ or “gs://my_bucket/my_file.csv”

      Each read instance should consist of exactly one read timestamp and one or more entity IDs identifying entities of the corresponding EntityTypes whose Features are requested.

      Each output instance contains Feature values of requested entities concatenated together as of the read time.

      An example read instance may be foo_entity_id, bar_entity_id, 2020-01-01T10:00:00.123Z.

      An example output instance may be foo_entity_id, bar_entity_id, 2020-01-01T10:00:00.123Z, foo_entity_feature1_value, bar_entity_feature2_value.

      Timestamp in each read instance must be millisecond-aligned.

      The columns can be in any order.

      Values in the timestamp column must use the RFC 3339 format, e.g. 2012-07-30T10:43:17.123Z.

    • pass_through_fields (List[str]) – Optional. When not empty, the specified fields in the read_instances source will be joined as-is in the output, in addition to those fields from the Featurestore Entity.

      For BigQuery source, the type of the pass-through values will be automatically inferred. For CSV source, the pass-through values will be passed as opaque bytes.

    • feature_destination_fields (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. A user defined dictionary to map a feature’s fully qualified resource name to its destination field name. If the destination field name is not defined, the feature ID will be used as its destination field name.

      Example

      feature_destination_fields = {

      ‘projects/123/locations/us-central1/featurestores/fs_id/entityTypes/et_id1/features/f_id11’: ‘foo’, ‘projects/123/locations/us-central1/featurestores/fs_id/entityTypes/et_id2/features/f_id22’: ‘bar’,

      }

    • start_time (timestamp_pb2.Timestamp) – Optional. Excludes Feature values with feature generation timestamp before this timestamp. If not set, retrieve oldest values kept in Feature Store. Timestamp, if present, must not have higher than millisecond precision.

    • serve_request_timeout (float) – Optional. The timeout for the serve request in seconds.

  • Returns

    The featurestore resource object batch read feature values from.

  • Return type

    Featurestore

  • Raises

    ValueError if gcs_destination_type is not supported. –

client_class()

alias of google.cloud.aiplatform.utils.FeaturestoreClientWithOverride

classmethod create(featurestore_id: str, online_store_fixed_node_count: Optional[int] = None, labels: Optional[Dict[str, str]] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None, request_metadata: Optional[Sequence[Tuple[str, str]]] = (), encryption_spec_key_name: Optional[str] = None, sync: bool = True, create_request_timeout: Optional[float] = None)

Creates a Featurestore resource.

Example Usage:

my_featurestore = aiplatform.Featurestore.create(

featurestore_id=’my_featurestore_id’,

)

  • Parameters

    • featurestore_id (str) – Required. The ID to use for this Featurestore, which will become the final component of the Featurestore’s resource name.

      This value may be up to 60 characters, and valid characters are [a-z0-9_]. The first character cannot be a number.

      The value must be unique within the project and location.

    • online_store_fixed_node_count (int) – Optional. Config for online serving resources. When not specified, no fixed node count for online serving. The number of nodes will not scale automatically but can be scaled manually by providing different values when updating.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The labels with user-defined metadata to organize your Featurestore. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information on and examples of labels. No more than 64 user labels can be associated with one Featurestore(System labels are excluded).” System reserved label keys are prefixed with “aiplatform.googleapis.com/” and are immutable.

    • project (str) – Optional. Project to create EntityType in. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to create EntityType in. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to create EntityTypes. Overrides credentials set in aiplatform.init.

    • request_metadata (Sequence[Tuple[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*]) – Optional. Strings which should be sent along with the request as metadata.

    • encryption_spec (str) – Optional. Customer-managed encryption key spec for data storage. If set, both of the online and offline data storage will be secured by this key.

    • sync (bool) – Optional. Whether to execute this creation synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

    • create_request_timeout (float) – Optional. The timeout for the create request in seconds.

  • Returns

    Featurestore - Featurestore resource object

create_entity_type(entity_type_id: str, description: Optional[str] = None, labels: Optional[Dict[str, str]] = None, request_metadata: Optional[Sequence[Tuple[str, str]]] = (), sync: bool = True, create_request_timeout: Optional[float] = None)

Creates an EntityType resource in this Featurestore.

Example Usage:

my_featurestore = aiplatform.Featurestore.create(

featurestore_id=’my_featurestore_id’

) my_entity_type = my_featurestore.create_entity_type(

entity_type_id=’my_entity_type_id’,

)

  • Parameters

    • entity_type_id (str) – Required. The ID to use for the EntityType, which will become the final component of the EntityType’s resource name.

      This value may be up to 60 characters, and valid characters are [a-z0-9_]. The first character cannot be a number.

      The value must be unique within a featurestore.

    • description (str) – Optional. Description of the EntityType.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The labels with user-defined metadata to organize your EntityTypes. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information on and examples of labels. No more than 64 user labels can be associated with one EntityType (System labels are excluded).” System reserved label keys are prefixed with “aiplatform.googleapis.com/” and are immutable.

    • request_metadata (Sequence[Tuple[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*]) – Optional. Strings which should be sent along with the request as metadata.

    • create_request_timeout (float) – Optional. The timeout for the create request in seconds.

    • sync (bool) – Optional. Whether to execute this creation synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

  • Returns

    featurestore.EntityType - EntityType resource object

property create_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was created.

delete(sync: bool = True, force: bool = False)

Deletes this Featurestore resource. If force is set to True, all entityTypes in this Featurestore will be deleted prior to featurestore deletion, and all features in each entityType will be deleted prior to each entityType deletion.

WARNING: This deletion is permanent.

  • Parameters

    • force (bool) – If set to true, any EntityTypes and Features for this Featurestore will also be deleted. (Otherwise, the request will only work if the Featurestore has no EntityTypes.)

    • sync (bool) – Whether to execute this deletion synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

delete_entity_types(entity_type_ids: List[str], sync: bool = True, force: bool = False)

Deletes entity_type resources in this Featurestore given their entity_type IDs. WARNING: This deletion is permanent.

  • Parameters

    • entity_type_ids (List[str]) – Required. The list of entity_type IDs to be deleted.

    • sync (bool) – Optional. Whether to execute this deletion synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

    • force (bool) – Optional. If force is set to True, all features in each entityType will be deleted prior to entityType deletion. Default is False.

property display_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Display name of this resource.

property encryption_spec(: Optional[[google.cloud.aiplatform_v1.types.encryption_spec.EncryptionSpec](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.EncryptionSpec)_ )

Customer-managed encryption key options for this Vertex AI resource.

If this is set, then all resources created by this Vertex AI resource will be encrypted with the provided encryption key.

property gca_resource(: [proto.message.Message](https://proto-plus-python.readthedocs.io/en/latest/reference/message.html#proto.message.Message )

The underlying resource proto representation.

get_entity_type(entity_type_id: str)

Retrieves an existing managed entityType in this Featurestore.

  • Parameters

    entity_type_id (str) – Required. The managed entityType resource ID in this Featurestore.

  • Returns

    featurestore.EntityType - The managed entityType resource object.

property labels(: Dict[str, str )

User-defined labels containing metadata about this resource.

Read more about labels at https://goo.gl/xmQnxf

classmethod list(filter: Optional[str] = None, order_by: Optional[str] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None, parent: Optional[str] = None)

List all instances of this Vertex AI Resource.

Example Usage:

aiplatform.BatchPredictionJobs.list(

filter=’state=”JOB_STATE_SUCCEEDED” AND display_name=”my_job”’,

)

aiplatform.Model.list(order_by=”create_time desc, display_name”)

  • Parameters

    • filter (str) – Optional. An expression for filtering the results of the request. For field names both snake_case and camelCase are supported.

    • order_by (str) – Optional. A comma-separated list of fields to order by, sorted in ascending order. Use “desc” after a field name for descending. Supported fields: display_name, create_time, update_time

    • project (str) – Optional. Project to retrieve list from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to retrieve list from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to retrieve list. Overrides credentials set in aiplatform.init.

    • parent (str) – Optional. The parent resource name if any to retrieve list from.

  • Returns

    List[VertexAiResourceNoun] - A list of SDK resource objects

list_entity_types(filter: Optional[str] = None, order_by: Optional[str] = None)

Lists existing managed entityType resources in this Featurestore.

Example Usage:

my_featurestore = aiplatform.Featurestore(

featurestore_name=’my_featurestore_id’,

) my_featurestore.list_entity_types()

  • Parameters

    • filter (str) – Optional. Lists the EntityTypes that match the filter expression. The following filters are supported:

      • create_time: Supports =, !=, <, >, >=, and <= comparisons. Values must be in RFC 3339 format.

      • update_time: Supports =, !=, <, >, >=, and <= comparisons. Values must be in RFC 3339 format.

      • labels: Supports key-value equality as well as key presence.

      Examples:

      • create_time > "2020-01-31T15:30:00.000000Z" OR update_time > "2020-01-31T15:30:00.000000Z" –> EntityTypes created or updated after 2020-01-31T15:30:00.000000Z.

      • labels.active = yes AND labels.env = prod –> EntityTypes having both (active: yes) and (env: prod) labels.

      • labels.env: \* –> Any EntityType which has a label with ‘env’ as the key.

    • order_by (str) – Optional. A comma-separated list of fields to order by, sorted in ascending order. Use “desc” after a field name for descending.

      Supported fields:

      • entity_type_id

      • create_time

      • update_time

  • Returns

    List[featurestore.EntityType] - A list of managed entityType resource objects.

property name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Name of this resource.

property resource_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Full qualified resource name.

to_dict()

Returns the resource proto as a dictionary.

update(labels: Optional[Dict[str, str]] = None, request_metadata: Optional[Sequence[Tuple[str, str]]] = (), update_request_timeout: Optional[float] = None)

Updates an existing managed featurestore resource.

Example Usage:

my_featurestore = aiplatform.Featurestore(

featurestore_name=’my_featurestore_id’,

) my_featurestore.update(

labels={‘update my key’: ‘update my value’},

)

  • Parameters

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The labels with user-defined metadata to organize your Featurestores. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information on and examples of labels. No more than 64 user labels can be associated with one Feature (System labels are excluded).” System reserved label keys are prefixed with “aiplatform.googleapis.com/” and are immutable.

    • request_metadata (Sequence[Tuple[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*]) – Optional. Strings which should be sent along with the request as metadata.

    • update_request_timeout (float) – Optional. The timeout for the update request in seconds.

  • Returns

    Featurestore - The updated featurestore resource object.

update_online_store(fixed_node_count: int, request_metadata: Optional[Sequence[Tuple[str, str]]] = (), update_request_timeout: Optional[float] = None)

Updates the online store of an existing managed featurestore resource.

Example Usage:

my_featurestore = aiplatform.Featurestore(

featurestore_name=’my_featurestore_id’,

) my_featurestore.update_online_store(

fixed_node_count=2,

)

  • Parameters

    • fixed_node_count (int) – Required. Config for online serving resources, can only update the node count to >= 1.

    • request_metadata (Sequence[Tuple[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*]) – Optional. Strings which should be sent along with the request as metadata.

    • update_request_timeout (float) – Optional. The timeout for the update request in seconds.

  • Returns

    Featurestore - The updated featurestore resource object.

property update_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was last updated.

wait()

Helper method that blocks until all futures are complete.

class google.cloud.aiplatform.HyperparameterTuningJob(display_name: str, custom_job: google.cloud.aiplatform.jobs.CustomJob, metric_spec: Dict[str, str], parameter_spec: Dict[str, google.cloud.aiplatform.hyperparameter_tuning._ParameterSpec], max_trial_count: int, parallel_trial_count: int, max_failed_trial_count: int = 0, search_algorithm: Optional[str] = None, measurement_selection: Optional[str] = 'best', project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None, labels: Optional[Dict[str, str]] = None, encryption_spec_key_name: Optional[str] = None)

Bases: google.cloud.aiplatform.jobs._RunnableJob, google.cloud.aiplatform.base.PreviewMixin

Vertex AI Hyperparameter Tuning Job.

Configures a HyperparameterTuning Job.

Example usage:

``

` from google.cloud.aiplatform import hyperparameter_tuning as hpt

worker_pool_specs = [

> {

>     “machine_spec”: {

>         “machine_type”: “n1-standard-4”,
>         “accelerator_type”: “NVIDIA_TESLA_K80”,
>         “accelerator_count”: 1,

>     },
>     “replica_count”: 1,
>     “container_spec”: {

>     > “image_uri”: container_image_uri,
>     > “command”: [],
>     > “args”: [],

>     },

> }

]

custom_job = aiplatform.CustomJob(

display_name=’my_job’,
worker_pool_specs=worker_pool_specs,
labels={‘my_key’: ‘my_value’},

)

hp_job = aiplatform.HyperparameterTuningJob(

display_name=’hp-test’,
custom_job=job,
metric_spec={

> ‘loss’: ‘minimize’,

},
parameter_spec={

> ‘lr’: hpt.DoubleParameterSpec(min=0.001, max=0.1, scale=’log’),
> ‘units’: hpt.IntegerParameterSpec(min=4, max=128, scale=’linear’),
> ‘activation’: hpt.CategoricalParameterSpec(values=[‘relu’, ‘selu’]),
> ‘batch_size’: hpt.DiscreteParameterSpec(values=[128, 256], scale=’linear’)

},
max_trial_count=128,
parallel_trial_count=8,
labels={‘my_key’: ‘my_value’},
)

hp_job.run()

print(hp_job.trials)

``

`

For more information on using hyperparameter tuning please visit: https://cloud.google.com/ai-platform-unified/docs/training/using-hyperparameter-tuning

  • Parameters

    • display_name (str) – Required. The user-defined name of the HyperparameterTuningJob. The name can be up to 128 characters long and can be consist of any UTF-8 characters.

    • custom_job (aiplatform.CustomJob) – Required. Configured CustomJob. The worker pool spec from this custom job applies to the CustomJobs created in all the trials. A persistent_resource_id can be specified on the custom job to be used when running this Hyperparameter Tuning job.

    • metric_spec – Dict[str, str] Required. Dictionary representing metrics to optimize. The dictionary key is the metric_id, which is reported by your training job, and the dictionary value is the optimization goal of the metric(‘minimize’ or ‘maximize’). example:

      metric_spec = {‘loss’: ‘minimize’, ‘accuracy’: ‘maximize’}

    • parameter_spec (Dict[str, **hyperparameter_tuning._ParameterSpec]) – Required. Dictionary representing parameters to optimize. The dictionary key is the metric_id, which is passed into your training job as a command line key word argument, and the dictionary value is the parameter specification of the metric.

      from google.cloud.aiplatform import hyperparameter_tuning as hpt

      parameter_spec={

      ‘decay’: hpt.DoubleParameterSpec(min=1e-7, max=1, scale=’linear’), ‘learning_rate’: hpt.DoubleParameterSpec(min=1e-7, max=1, scale=’linear’) ‘batch_size’: hpt.DiscreteParamterSpec(values=[4, 8, 16, 32, 64, 128], scale=’linear’)

      }

      Supported parameter specifications can be found until aiplatform.hyperparameter_tuning. These parameter specification are currently supported: DoubleParameterSpec, IntegerParameterSpec, CategoricalParameterSpace, DiscreteParameterSpec

    • max_trial_count (int) – Required. The desired total number of Trials.

    • parallel_trial_count (int) – Required. The desired number of Trials to run in parallel.

    • max_failed_trial_count (int) – Optional. The number of failed Trials that need to be seen before failing the HyperparameterTuningJob. If set to 0, Vertex AI decides how many Trials must fail before the whole job fails.

    • search_algorithm (str) – The search algorithm specified for the Study. Accepts one of the following:

      None - If you do not specify an algorithm, your job uses the default Vertex AI algorithm. The default algorithm applies Bayesian optimization to arrive at the optimal solution with a more effective search over the parameter space.

      ’grid’ - A simple grid search within the feasible space. This option is particularly useful if you want to specify a quantity of trials that is greater than the number of points in the feasible space. In such cases, if you do not specify a grid search, the Vertex AI default algorithm may generate duplicate suggestions. To use grid search, all parameter specs must be of type IntegerParameterSpec, CategoricalParameterSpace, or DiscreteParameterSpec.

      ’random’ - A simple random search within the feasible space.

    • measurement_selection (str) – This indicates which measurement to use if/when the service automatically selects the final measurement from previously reported intermediate measurements.

      Accepts: ‘best’, ‘last’

      Choose this based on two considerations: A) Do you expect your measurements to monotonically improve? If so, choose ‘last’. On the other hand, if you’re in a situation where your system can “over-train” and you expect the performance to get better for a while but then start declining, choose ‘best’. B) Are your measurements significantly noisy and/or irreproducible? If so, ‘best’ will tend to be over-optimistic, and it may be better to choose ‘last’. If both or neither of (A) and (B) apply, it doesn’t matter which selection type is chosen.

    • project (str) – Optional. Project to run the HyperparameterTuningjob in. Overrides project set in aiplatform.init.

    • location (str) – Optional. Location to run the HyperparameterTuning in. Overrides location set in aiplatform.init.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to run call HyperparameterTuning service. Overrides credentials set in aiplatform.init.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The labels with user-defined metadata to organize HyperparameterTuningJobs. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information and examples of labels.

    • encryption_spec_key_name (str) – Optional. Customer-managed encryption key options for a HyperparameterTuningJob. If this is set, then all resources created by the HyperparameterTuningJob will be encrypted with the provided encryption key.

cancel()

Cancels this Job.

Success of cancellation is not guaranteed. Use Job.state property to verify if cancellation was successful.

client_class()

alias of google.cloud.aiplatform.utils.JobClientWithOverride

property create_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was created.

delete(sync: bool = True)

Deletes this Vertex AI resource. WARNING: This deletion is permanent.

  • Parameters

    sync (bool) – Whether to execute this deletion synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

property display_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Display name of this resource.

done()

Method indicating whether a job has completed.

  • Returns

    True if the job has completed.

property encryption_spec(: Optional[[google.cloud.aiplatform_v1.types.encryption_spec.EncryptionSpec](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.EncryptionSpec)_ )

Customer-managed encryption key options for this Vertex AI resource.

If this is set, then all resources created by this Vertex AI resource will be encrypted with the provided encryption key.

property end_time(: Optional[datetime.datetime )

Time when the Job resource entered the JOB_STATE_SUCCEEDED, JOB_STATE_FAILED, or JOB_STATE_CANCELLED state.

property error(: Optional[google.rpc.status_pb2.Status )

Detailed error info for this Job resource. Only populated when the Job’s state is JOB_STATE_FAILED or JOB_STATE_CANCELLED.

property gca_resource(: [proto.message.Message](https://proto-plus-python.readthedocs.io/en/latest/reference/message.html#proto.message.Message )

The underlying resource proto representation.

classmethod get(resource_name: str, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Get a Vertex AI Job for the given resource_name.

  • Parameters

    • resource_name (str) – Required. A fully-qualified resource name or ID.

    • project (str) – Optional. project to retrieve dataset from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. location to retrieve dataset from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Custom credentials to use to upload this model. Overrides credentials set in aiplatform.init.

  • Returns

    A Vertex AI Job.

property labels(: Dict[str, str )

User-defined labels containing metadata about this resource.

Read more about labels at https://goo.gl/xmQnxf

classmethod list(filter: Optional[str] = None, order_by: Optional[str] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

List all instances of this Job Resource.

Example Usage:

aiplatform.BatchPredictionJobs.list(

filter=’state=”JOB_STATE_SUCCEEDED” AND display_name=”my_job”’,

)

  • Parameters

    • filter (str) – Optional. An expression for filtering the results of the request. For field names both snake_case and camelCase are supported.

    • order_by (str) – Optional. A comma-separated list of fields to order by, sorted in ascending order. Use “desc” after a field name for descending. Supported fields: display_name, create_time, update_time

    • project (str) – Optional. Project to retrieve list from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to retrieve list from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to retrieve list. Overrides credentials set in aiplatform.init.

  • Returns

    List[VertexAiResourceNoun] - A list of Job resource objects.

property name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Name of this resource.

property network(: Optional[str )

The full name of the Google Compute Engine network to which this HyperparameterTuningJob should be peered.

Takes the format projects/{project}/global/networks/{network}. Where {project} is a project number, as in 12345, and {network} is a network name.

Private services access must already be configured for the network. If left unspecified, the HyperparameterTuningJob is not peered with any network.

property preview(: google.cloud.aiplatform.base.PreviewClas )

Exposes features available in preview for this class.

property resource_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Full qualified resource name.

run(service_account: Optional[str] = None, network: Optional[str] = None, timeout: Optional[int] = None, restart_job_on_worker_restart: bool = False, enable_web_access: bool = False, tensorboard: Optional[str] = None, sync: bool = True, create_request_timeout: Optional[float] = None, disable_retries: bool = False)

Run this configured CustomJob.

  • Parameters

    • service_account (str) – Optional. Specifies the service account for workload run-as account. Users submitting jobs must have act-as permission on this run-as account.

    • network (str) – Optional. The full name of the Compute Engine network to which the job should be peered. For example, projects/12345/global/networks/myVPC. Private services access must already be configured for the network. If left unspecified, the network set in aiplatform.init will be used. Otherwise, the job is not peered with any network.

    • timeout (int) – Optional. The maximum job running time in seconds. The default is 7 days.

    • restart_job_on_worker_restart (bool) – Restarts the entire CustomJob if a worker gets restarted. This feature can be used by distributed training jobs that are not resilient to workers leaving and joining a job.

    • enable_web_access (bool) – Whether you want Vertex AI to enable interactive shell access to training containers. https://cloud.google.com/vertex-ai/docs/training/monitor-debug-interactive-shell

    • tensorboard (str) – Optional. The name of a Vertex AI [Tensorboard][google.cloud.aiplatform.v1beta1.Tensorboard] resource to which this CustomJob will upload Tensorboard logs. Format: projects/{project}/locations/{location}/tensorboards/{tensorboard}

      The training script should write Tensorboard to following Vertex AI environment variable:

      AIP_TENSORBOARD_LOG_DIR

      service_account is required with provided tensorboard. For more information on configuring your service account please visit: https://cloud.google.com/vertex-ai/docs/experiments/tensorboard-training

    • sync (bool) – Whether to execute this method synchronously. If False, this method will unblock and it will be executed in a concurrent Future.

    • create_request_timeout (float) – Optional. The timeout for the create request in seconds.

    • disable_retries (bool) – Indicates if the job should retry for internal errors after the job starts running. If True, overrides restart_job_on_worker_restart to False.

property start_time(: Optional[datetime.datetime )

Time when the Job resource entered the JOB_STATE_RUNNING for the first time.

property state(: [google.cloud.aiplatform_v1.types.job_state.JobState](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.JobState_ )

Fetch Job again and return the current JobState.

  • Returns

    Enum that describes the state of a Vertex AI job.

  • Return type

    state (job_state.JobState)

to_dict()

Returns the resource proto as a dictionary.

property update_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was last updated.

wait()

Helper method that blocks until all futures are complete.

wait_for_completion()

Waits for job to complete.

wait_for_resource_creation()

Waits until resource has been created.

property web_access_uris(: Dict[str, Union[str, Dict[str, str]] )

Fetch the runnable job again and return the latest web access uris.

  • Returns

    Web access uris of the runnable job.

  • Return type

    (Dict[str, Union[str, Dict[str, str]]])

class google.cloud.aiplatform.ImageDataset(dataset_name: str, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Bases: google.cloud.aiplatform.datasets.dataset._Dataset

A managed image dataset resource for Vertex AI.

Use this class to work with a managed image dataset. To create a managed image dataset, you need a datasource file in CSV format and a schema file in YAML format. A schema is optional for a custom model. You put the CSV file and the schema into Cloud Storage buckets.

Use image data for the following objectives:

  • Single-label classification. For more information, see

Prepare image training data for single-label classification. * Multi-label classification. For more information, see Prepare image training data for multi-label classification. * Object detection. For more information, see [Prepare image training data

for object detection](https://cloud.google.com/vertex-ai/docs/image-data/object-detection/prepare-data).

The following code shows you how to create an image dataset by importing data from a CSV datasource file and a YAML schema file. The schema file you use depends on whether your image dataset is used for single-label classification, multi-label classification, or object detection.

``

`

py my_dataset = aiplatform.ImageDataset.create(

display_name=”my-image-dataset”, gcs_source=[‘gs://path/to/my/image-dataset.csv’], import_schema_uri=[‘gs://path/to/my/schema.yaml’]

Retrieves an existing managed dataset given a dataset name or ID.

  • Parameters

    • dataset_name (str) – Required. A fully-qualified dataset resource name or dataset ID. Example: “projects/123/locations/us-central1/datasets/456” or “456” when project and location are initialized or passed.

    • project (str) – Optional project to retrieve dataset from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional location to retrieve dataset from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Custom credentials to use to retrieve this Dataset. Overrides credentials set in aiplatform.init.

client_class()

alias of google.cloud.aiplatform.utils.DatasetClientWithOverride

classmethod create(display_name: Optional[str] = None, gcs_source: Optional[Union[str, Sequence[str]]] = None, import_schema_uri: Optional[str] = None, data_item_labels: Optional[Dict] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None, request_metadata: Optional[Sequence[Tuple[str, str]]] = (), labels: Optional[Dict[str, str]] = None, encryption_spec_key_name: Optional[str] = None, sync: bool = True, create_request_timeout: Optional[float] = None)

Creates a new image dataset.

Optionally imports data into the dataset when a source and import_schema_uri are passed in.

  • Parameters

    • display_name (str) – Optional. The user-defined name of the dataset. The name must contain 128 or fewer UTF-8 characters.

    • gcs_source (Union[str, **Sequence[str]]) – Optional. The URI to one or more Google Cloud Storage buckets that contain your datasets. For example, str: “gs://bucket/file.csv” or Sequence[str]: [“gs://bucket/file1.csv”, “gs://bucket/file2.csv”].

    • import_schema_uri (str) – Optional. A URI for a YAML file stored in Cloud Storage that describes the import schema used to validate the dataset. The schema is an OpenAPI 3.0.2 Schema object.

    • data_item_labels (Dict) – Optional. A dictionary of label information. Each dictionary item contains a label and a label key. Each image in the dataset includes one dictionary of label information. If a data item is added or merged into a dataset, and that data item contains an image that’s identical to an image that’s already in the dataset, then the data items are merged. If two identical labels are detected during the merge, each with a different label key, then one of the label and label key dictionary items is randomly chosen to be into the merged data item. Images and documents are compared using their binary data (bytes), not on their content. If annotation labels are referenced in a schema specified by the import_schema_url parameter, then the labels in the data_item_labels dictionary are overriden by the annotations.

    • project (str) – Optional. The name of the Google Cloud project to which this ImageDataset is uploaded. This overrides the project that was set by aiplatform.init.

    • location (str) – Optional. The Google Cloud region where this dataset is uploaded. This region overrides the region that was set by aiplatform.init.

    • credentials (auth_credentials.Credentials) – Optional. The credentials that are used to upload the ImageDataset. These credentials override the credentials set by aiplatform.init.

    • request_metadata (Sequence[Tuple[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*]) – Optional. Strings that contain metadata that’s sent with the request.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. Labels with user-defined metadata to organize your Vertex AI Tensorboards. The maximum length of a key and of a value is 64 unicode characters. Labels and keys can contain only lowercase letters, numeric characters, underscores, and dashes. International characters are allowed. No more than 64 user labels can be associated with one Tensorboard (system labels are excluded). For more information and examples of using labels, see Using labels to organize Google Cloud Platform resources. System reserved label keys are prefixed with aiplatform.googleapis.com/ and are immutable.

    • encryption_spec_key_name (Optional[str]) – Optional. The Cloud KMS resource identifier of the customer managed encryption key that’s used to protect the dataset. The format of the key is projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key. The key needs to be in the same region as where the compute resource is created.

      If encryption_spec_key_name is set, this image dataset and all of its sub-resources are secured by this key.

      This encryption_spec_key_name overrides the encryption_spec_key_name set by aiplatform.init.

    • sync (bool) – If true, the create method creates an image dataset synchronously. If false, the create method creates an image dataset asynchronously.

    • create_request_timeout (float) – Optional. The number of seconds for the timeout of the create request.

  • Returns

    An instantiated representation of the managed ImageDataset resource.

  • Return type

    image_dataset (ImageDataset)

property create_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was created.

delete(sync: bool = True)

Deletes this Vertex AI resource. WARNING: This deletion is permanent.

  • Parameters

    sync (bool) – Whether to execute this deletion synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

property display_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Display name of this resource.

property encryption_spec(: Optional[[google.cloud.aiplatform_v1.types.encryption_spec.EncryptionSpec](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.EncryptionSpec)_ )

Customer-managed encryption key options for this Vertex AI resource.

If this is set, then all resources created by this Vertex AI resource will be encrypted with the provided encryption key.

export_data(output_dir: str)

Exports data to output dir to GCS.

  • Parameters

    output_dir (str) – Required. The Google Cloud Storage location where the output is to be written to. In the given directory a new directory will be created with name: export-data-<dataset-display-name>-<timestamp-of-export-call> where timestamp is in YYYYMMDDHHMMSS format. All export output will be written into that directory. Inside that directory, annotations with the same schema will be grouped into sub directories which are named with the corresponding annotations’ schema title. Inside these sub directories, a schema.yaml will be created to describe the output format.

    If the uri doesn’t end with ‘/’, a ‘/’ will be automatically appended. The directory is created if it doesn’t exist.

  • Returns

    All of the files that are exported in this export operation.

  • Return type

    exported_files (Sequence[str])

export_data_for_custom_training(output_dir: str, annotation_filter: Optional[str] = None, saved_query_id: Optional[str] = None, annotation_schema_uri: Optional[str] = None, split: Optional[Union[Dict[str, str], Dict[str, float]]] = None)

Exports data to output dir to GCS for custom training use case.

Example annotation_schema_uri (image classification): gs://google-cloud-aiplatform/schema/dataset/annotation/image_classification_1.0.0.yaml

Example split (filter split): {

“training_filter”: “labels.aiplatform.googleapis.com/ml_use=training”, “validation_filter”: “labels.aiplatform.googleapis.com/ml_use=validation”, “test_filter”: “labels.aiplatform.googleapis.com/ml_use=test”,

} Example split (fraction split): {

“training_fraction”: 0.7, “validation_fraction”: 0.2, “test_fraction”: 0.1,

}

  • Parameters

    • output_dir (str) – Required. The Google Cloud Storage location where the output is to be written to. In the given directory a new directory will be created with name: export-data-<dataset-display-name>-<timestamp-of-export-call> where timestamp is in YYYYMMDDHHMMSS format. All export output will be written into that directory. Inside that directory, annotations with the same schema will be grouped into sub directories which are named with the corresponding annotations’ schema title. Inside these sub directories, a schema.yaml will be created to describe the output format.

      If the uri doesn’t end with ‘/’, a ‘/’ will be automatically appended. The directory is created if it doesn’t exist.

    • annotation_filter (str) – Optional. An expression for filtering what part of the Dataset is to be exported. Only Annotations that match this filter will be exported. The filter syntax is the same as in [ListAnnotations][DatasetService.ListAnnotations].

    • saved_query_id (str) – Optional. The ID of a SavedQuery (annotation set) under this Dataset used for filtering Annotations for training.

      Only used for custom training data export use cases. Only applicable to Datasets that have SavedQueries.

      Only Annotations that are associated with this SavedQuery are used in respectively training. When used in conjunction with annotations_filter, the Annotations used for training are filtered by both saved_query_id and annotations_filter.

      Only one of saved_query_id and annotation_schema_uri should be specified as both of them represent the same thing: problem type.

    • annotation_schema_uri (str) – Optional. The Cloud Storage URI that points to a YAML file describing the annotation schema. The schema is defined as an OpenAPI 3.0.2 Schema Object. The schema files that can be used here are found in gs://google-cloud-aiplatform/schema/dataset/annotation/, note that the chosen schema must be consistent with metadata_schema_uri of this Dataset.

      Only used for custom training data export use cases. Only applicable if this Dataset that have DataItems and Annotations.

      Only Annotations that both match this schema and belong to DataItems not ignored by the split method are used in respectively training, validation or test role, depending on the role of the DataItem they are on.

      When used in conjunction with annotations_filter, the Annotations used for training are filtered by both annotations_filter and annotation_schema_uri.

    • split (Union[Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)], **Dict[[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str), [float](https://python.readthedocs.io/en/latest/library/functions.html#float)]*]) – The instructions how the export data should be split between the training, validation and test sets.

  • Returns

    Response message for DatasetService.ExportData in Dictionary format.

  • Return type

    export_data_response (Dict)

property gca_resource(: [proto.message.Message](https://proto-plus-python.readthedocs.io/en/latest/reference/message.html#proto.message.Message )

The underlying resource proto representation.

import_data(gcs_source: Union[str, Sequence[str]], import_schema_uri: str, data_item_labels: Optional[Dict] = None, sync: bool = True, import_request_timeout: Optional[float] = None)

Upload data to existing managed dataset.

  • Parameters

    • gcs_source (Union[str, **Sequence[str]]) – Required. Google Cloud Storage URI(-s) to the input file(s). May contain wildcards. For more information on wildcards, see https://cloud.google.com/storage/docs/gsutil/addlhelp/WildcardNames. .. rubric:: Examples

      str: “gs://bucket/file.csv” Sequence[str]: [“gs://bucket/file1.csv”, “gs://bucket/file2.csv”]

    • import_schema_uri (str) – Required. Points to a YAML file stored on Google Cloud Storage describing the import format. Validation will be done against the schema. The schema is defined as an OpenAPI 3.0.2 Schema Object.

    • data_item_labels (Dict) – Labels that will be applied to newly imported DataItems. If an identical DataItem as one being imported already exists in the Dataset, then these labels will be appended to these of the already existing one, and if labels with identical key is imported before, the old label value will be overwritten. If two DataItems are identical in the same import data operation, the labels will be combined and if key collision happens in this case, one of the values will be picked randomly. Two DataItems are considered identical if their content bytes are identical (e.g. image bytes or pdf bytes). These labels will be overridden by Annotation labels specified inside index file referenced by import_schema_uri, e.g. jsonl file. This arg is not for specifying the annotation name or the training target of your data, but for some global labels of the dataset. E.g., ‘data_item_labels={“aiplatform.googleapis.com/ml_use”:”training”}’ specifies that all the uploaded data are used for training.

    • sync (bool) – Whether to execute this method synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

    • import_request_timeout (float) – Optional. The timeout for the import request in seconds.

  • Returns

    Instantiated representation of the managed dataset resource.

  • Return type

    dataset (Dataset)

property labels(: Dict[str, str )

User-defined labels containing metadata about this resource.

Read more about labels at https://goo.gl/xmQnxf

classmethod list(filter: Optional[str] = None, order_by: Optional[str] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

List all instances of this Dataset resource.

Example Usage:

aiplatform.TabularDataset.list(

filter=’labels.my_key=”my_value”’,
order_by=’display_name’

)

  • Parameters

    • filter (str) – Optional. An expression for filtering the results of the request. For field names both snake_case and camelCase are supported.

    • order_by (str) – Optional. A comma-separated list of fields to order by, sorted in ascending order. Use “desc” after a field name for descending. Supported fields: display_name, create_time, update_time

    • project (str) – Optional. Project to retrieve list from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to retrieve list from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to retrieve list. Overrides credentials set in aiplatform.init.

  • Returns

    List[base.VertexAiResourceNoun] - A list of Dataset resource objects

property metadata_schema_uri(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

The metadata schema uri of this dataset resource.

property name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Name of this resource.

property resource_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Full qualified resource name.

to_dict()

Returns the resource proto as a dictionary.

update(*, display_name: Optional[str] = None, labels: Optional[Dict[str, str]] = None, description: Optional[str] = None, update_request_timeout: Optional[float] = None)

Update the dataset.

Updatable fields:


    * `display_name`


    * `description`


    * `labels`

  • Parameters

    • display_name (str) – Optional. The user-defined name of the Dataset. The name can be up to 128 characters long and can be consist of any UTF-8 characters.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. Labels with user-defined metadata to organize your Tensorboards. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. No more than 64 user labels can be associated with one Tensorboard (System labels are excluded). See https://goo.gl/xmQnxf for more information and examples of labels. System reserved label keys are prefixed with “aiplatform.googleapis.com/” and are immutable.

    • description (str) – Optional. The description of the Dataset.

    • update_request_timeout (float) – Optional. The timeout for the update request in seconds.

  • Returns

    Updated dataset.

  • Return type

    dataset (Dataset)

property update_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was last updated.

wait()

Helper method that blocks until all futures are complete.

class google.cloud.aiplatform.MatchingEngineIndex(index_name: str, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Bases: google.cloud.aiplatform.base.VertexAiResourceNounWithFutureManager

Matching Engine index resource for Vertex AI.

Retrieves an existing index given an index name or ID.

Example Usage:

my_index = aiplatform.MatchingEngineIndex(

index_name=’projects/123/locations/us-central1/indexes/my_index_id’

) or my_index = aiplatform.MatchingEngineIndex(

index_name=’my_index_id’

)

  • Parameters

    • index_name (str) – Required. A fully-qualified index resource name or a index ID. Example: “projects/123/locations/us-central1/indexes/my_index_id” or “my_index_id” when project and location are initialized or passed.

    • project (str) – Optional. Project to retrieve index from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to retrieve index from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to retrieve this Index. Overrides credentials set in aiplatform.init.

client_class()

alias of google.cloud.aiplatform.utils.IndexClientWithOverride

classmethod create_brute_force_index(display_name: str, contents_delta_uri: Optional[str] = None, dimensions: Optional[int] = None, distance_measure_type: Optional[google.cloud.aiplatform.matching_engine.matching_engine_index_config.DistanceMeasureType] = None, feature_norm_type: Optional[google.cloud.aiplatform.matching_engine.matching_engine_index_config.FeatureNormType] = None, description: Optional[str] = None, labels: Optional[Dict[str, str]] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None, request_metadata: Optional[Sequence[Tuple[str, str]]] = (), sync: bool = True, index_update_method: Optional[str] = None, encryption_spec_key_name: Optional[str] = None, create_request_timeout: Optional[float] = None, shard_size: Optional[str] = None)

Creates a MatchingEngineIndex resource that uses the brute force algorithm.

Example Usage:

my_index = aiplatform.Index.create_brute_force_index(

display_name=”my_display_name”,
contents_delta_uri=”gs://my_bucket/embeddings”,
dimensions=1,
approximate_neighbors_count=150,
distance_measure_type=”SQUARED_L2_DISTANCE”,
description=”my description”,
labels={ “label_name”: “label_value” },

)

  • Parameters

    • display_name (str) – Required. The display name of the Index. The name can be up to 128 characters long and can be consist of any UTF-8 characters.

    • contents_delta_uri (str) – Optional. Allows inserting, updating or deleting the contents of the Matching Engine Index. The string must be a valid Google Cloud Storage directory path. If this field is set when calling IndexService.UpdateIndex, then no other Index field can be also updated as part of the same call. The expected structure and format of the files this URI points to is described at https://cloud.google.com/vertex-ai/docs/vector-search/setup/format-structure

    • dimensions (int) – Required. The number of dimensions of the input vectors.

    • distance_measure_type (matching_engine_index_config.DistanceMeasureType) – Optional. The distance measure used in nearest neighbor search.

    • feature_norm_type (matching_engine_index_config.FeatureNormType) – Optional. The feature norm type used in nearest neighbor search.

    • description (str) – Optional. The description of the Index.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The labels with user-defined metadata to organize your Index. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information on and examples of labels. No more than 64 user labels can be associated with one Index(System labels are excluded).” System reserved label keys are prefixed with “aiplatform.googleapis.com/” and are immutable.

    • project (str) – Optional. Project to create EntityType in. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to create EntityType in. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to create EntityTypes. Overrides credentials set in aiplatform.init.

    • request_metadata (Sequence[Tuple[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*]) – Optional. Strings which should be sent along with the request as metadata.

    • sync (bool) – Optional. Whether to execute this creation synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

    • index_update_method (str) – Optional. The update method to use with this index. Choose stream_update or batch_update. If not set, batch update will be used by default.

    • encryption_spec_key_name (str) – Optional. The Cloud KMS resource identifier of the customer managed encryption key used to protect the index. Has the form: projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key. The key needs to be in the same region as where the compute resource is created.

      If set, this index and all sub-resources of this index will be secured by this key. The key needs to be in the same region as where the index is created.

    • create_request_timeout (float) – Optional. The timeout for the request in seconds.

    • shard_size (str) – Optional. The size of each shard. Index will get resharded based on specified shard size. During serving, each shard will be served on a separate node and will scale independently.

      If not set, shard size is default to SHARD_SIZE_MEDIUM.

      Choose one of the following:

      SHARD_SIZE_SMALL SHARD_SIZE_MEDIUM SHARD_SIZE_LARGE

  • Returns

    MatchingEngineIndex - Index resource object

property create_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was created.

classmethod create_tree_ah_index(display_name: str, contents_delta_uri: Optional[str] = None, dimensions: Optional[int] = None, approximate_neighbors_count: Optional[int] = None, leaf_node_embedding_count: Optional[int] = None, leaf_nodes_to_search_percent: Optional[float] = None, distance_measure_type: Optional[google.cloud.aiplatform.matching_engine.matching_engine_index_config.DistanceMeasureType] = None, description: Optional[str] = None, labels: Optional[Dict[str, str]] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None, request_metadata: Optional[Sequence[Tuple[str, str]]] = (), sync: bool = True, index_update_method: Optional[str] = None, encryption_spec_key_name: Optional[str] = None, create_request_timeout: Optional[float] = None, shard_size: Optional[str] = None, feature_norm_type: Optional[google.cloud.aiplatform.matching_engine.matching_engine_index_config.FeatureNormType] = None)

Creates a MatchingEngineIndex resource that uses the tree-AH algorithm.

Example Usage:

my_index = aiplatform.Index.create_tree_ah_index(

display_name=”my_display_name”,
contents_delta_uri=”gs://my_bucket/embeddings”,
dimensions=1,
approximate_neighbors_count=150,
distance_measure_type=”SQUARED_L2_DISTANCE”,
leaf_node_embedding_count=100,
leaf_nodes_to_search_percent=50,
description=”my description”,
labels={ “label_name”: “label_value” },

)

  • Parameters

    • display_name (str) – Required. The display name of the Index. The name can be up to 128 characters long and can be consist of any UTF-8 characters.

    • contents_delta_uri (str) – Optional. Allows inserting, updating or deleting the contents of the Matching Engine Index. The string must be a valid Google Cloud Storage directory path. If this field is set when calling IndexService.UpdateIndex, then no other Index field can be also updated as part of the same call. The expected structure and format of the files this URI points to is described at https://cloud.google.com/vertex-ai/docs/vector-search/setup/format-structure

    • dimensions (int) – Required. The number of dimensions of the input vectors.

    • approximate_neighbors_count (int) – Required. The default number of neighbors to find via approximate search before exact reordering is performed. Exact reordering is a procedure where results returned by an approximate search algorithm are reordered via a more expensive distance computation.

    • leaf_node_embedding_count (int) – Optional. Number of embeddings on each leaf node. The default value is 1000 if not set.

    • leaf_nodes_to_search_percent (float) – Optional. The default percentage of leaf nodes that any query may be searched. Must be in range 1-100, inclusive. The default value is 10 (means 10%) if not set.

    • distance_measure_type (matching_engine_index_config.DistanceMeasureType) – Optional. The distance measure used in nearest neighbor search.

    • feature_norm_type (matching_engine_index_config.FeatureNormType) – Optional. The feature norm type used in nearest neighbor search.

    • description (str) – Optional. The description of the Index.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The labels with user-defined metadata to organize your Index. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information on and examples of labels. No more than 64 user labels can be associated with one Index(System labels are excluded).” System reserved label keys are prefixed with “aiplatform.googleapis.com/” and are immutable.

    • project (str) – Optional. Project to create EntityType in. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to create EntityType in. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to create EntityTypes. Overrides credentials set in aiplatform.init.

    • request_metadata (Sequence[Tuple[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*]) – Optional. Strings which should be sent along with the request as metadata.

    • sync (bool) – Optional. Whether to execute this creation synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

    • index_update_method (str) – Optional. The update method to use with this index. Choose STREAM_UPDATE or BATCH_UPDATE. If not set, batch update will be used by default.

    • encryption_spec_key_name (str) – Optional. The Cloud KMS resource identifier of the customer managed encryption key used to protect the index. Has the form: projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key. The key needs to be in the same region as where the compute resource is created.

      If set, this index and all sub-resources of this index will be secured by this key. The key needs to be in the same region as where the index is created.

    • create_request_timeout (float) – Optional. The timeout for the request in seconds.

    • shard_size (str) – Optional. The size of each shard. Index will get resharded based on specified shard size. During serving, each shard will be served on a separate node and will scale independently.

      Choose one of the following:

      SHARD_SIZE_SMALL SHARD_SIZE_MEDIUM SHARD_SIZE_LARGE

  • Returns

    MatchingEngineIndex - Index resource object

delete(sync: bool = True)

Deletes this Vertex AI resource. WARNING: This deletion is permanent.

  • Parameters

    sync (bool) – Whether to execute this deletion synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

property deployed_indexes(: List[[google.cloud.aiplatform_v1.types.deployed_index_ref.DeployedIndexRef](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.DeployedIndexRef)_ )

Returns a list of deployed index references that originate from this index.

  • Returns

    List[gca_matching_engine_deployed_index_ref.DeployedIndexRef] - Deployed index references

property description(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Description of the index.

property display_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Display name of this resource.

property encryption_spec(: Optional[[google.cloud.aiplatform_v1.types.encryption_spec.EncryptionSpec](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.EncryptionSpec)_ )

Customer-managed encryption key options for this Vertex AI resource.

If this is set, then all resources created by this Vertex AI resource will be encrypted with the provided encryption key.

property gca_resource(: [proto.message.Message](https://proto-plus-python.readthedocs.io/en/latest/reference/message.html#proto.message.Message )

The underlying resource proto representation.

property labels(: Dict[str, str )

User-defined labels containing metadata about this resource.

Read more about labels at https://goo.gl/xmQnxf

classmethod list(filter: Optional[str] = None, order_by: Optional[str] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None, parent: Optional[str] = None)

List all instances of this Vertex AI Resource.

Example Usage:

aiplatform.BatchPredictionJobs.list(

filter=’state=”JOB_STATE_SUCCEEDED” AND display_name=”my_job”’,

)

aiplatform.Model.list(order_by=”create_time desc, display_name”)

  • Parameters

    • filter (str) – Optional. An expression for filtering the results of the request. For field names both snake_case and camelCase are supported.

    • order_by (str) – Optional. A comma-separated list of fields to order by, sorted in ascending order. Use “desc” after a field name for descending. Supported fields: display_name, create_time, update_time

    • project (str) – Optional. Project to retrieve list from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to retrieve list from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to retrieve list. Overrides credentials set in aiplatform.init.

    • parent (str) – Optional. The parent resource name if any to retrieve list from.

  • Returns

    List[VertexAiResourceNoun] - A list of SDK resource objects

property name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Name of this resource.

remove_datapoints(datapoint_ids: Sequence[str])

Remove datapoints for this index.

  • Parameters

    datapoints_ids (Sequence[str]) – Required. The list of datapoints ids to be deleted.

  • Returns

    MatchingEngineIndex - Index resource object

property resource_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Full qualified resource name.

to_dict()

Returns the resource proto as a dictionary.

update_embeddings(contents_delta_uri: str, is_complete_overwrite: Optional[bool] = None, request_metadata: Optional[Sequence[Tuple[str, str]]] = (), update_request_timeout: Optional[float] = None)

Updates the embeddings for this index.

  • Parameters

    • contents_delta_uri (str) – Required. Allows inserting, updating or deleting the contents of the Matching Engine Index. The string must be a valid Google Cloud Storage directory path. If this field is set when calling IndexService.UpdateIndex, then no other Index field can be also updated as part of the same call. The expected structure and format of the files this URI points to is described at https://cloud.google.com/vertex-ai/docs/vector-search/setup/format-structure

    • is_complete_overwrite (bool) – Optional. If this field is set together with contentsDeltaUri when calling IndexService.UpdateIndex, then existing content of the Index will be replaced by the data from the contentsDeltaUri.

    • request_metadata (Sequence[Tuple[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*]) – Optional. Strings which should be sent along with the request as metadata.

    • update_request_timeout (float) – Optional. The timeout for the request in seconds.

  • Returns

    MatchingEngineIndex - The updated index resource object.

update_metadata(display_name: Optional[str] = None, description: Optional[str] = None, labels: Optional[Dict[str, str]] = None, request_metadata: Optional[Sequence[Tuple[str, str]]] = (), update_request_timeout: Optional[float] = None)

Updates the metadata for this index.

  • Parameters

    • display_name (str) – Optional. The display name of the Index. The name can be up to 128 characters long and can be consist of any UTF-8 characters.

    • description (str) – Optional. The description of the Index.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The labels with user-defined metadata to organize your Indexs. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information on and examples of labels. No more than 64 user labels can be associated with one Index (System labels are excluded).” System reserved label keys are prefixed with “aiplatform.googleapis.com/” and are immutable.

    • request_metadata (Sequence[Tuple[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*]) – Optional. Strings which should be sent along with the request as metadata.

    • update_request_timeout (float) – Optional. The timeout for the request in seconds.

  • Returns

    MatchingEngineIndex - The updated index resource object.

property update_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was last updated.

upsert_datapoints(datapoints: Sequence[google.cloud.aiplatform_v1.types.index.IndexDatapoint], update_mask: Optional[Sequence[str]] = None)

Upsert datapoints to this index.

  • Parameters

    • datapoints (Sequence[gca_matching_engine_index.IndexDatapoint]) – Required. Datapoints to be upserted to this index.

    • update_mask (Sequence[str]) – Optional. Update mask is used to specify the fields to be overwritten in the datapoints by the update. The fields specified in the update_mask are relative to each IndexDatapoint inside datapoints, not the full request. Updatable fields:

      Use all_restricts to update both restricts and numeric_restricts.

  • Returns

    MatchingEngineIndex - Index resource object

wait()

Helper method that blocks until all futures are complete.

class google.cloud.aiplatform.MatchingEngineIndexEndpoint(index_endpoint_name: str, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Bases: google.cloud.aiplatform.base.VertexAiResourceNounWithFutureManager

Matching Engine index endpoint resource for Vertex AI.

Retrieves an existing index endpoint given a name or ID.

Example Usage:

my_index_endpoint = aiplatform.MatchingEngineIndexEndpoint(

index_endpoint_name=’projects/123/locations/us-central1/index_endpoint/my_index_id’

) or my_index_endpoint = aiplatform.MatchingEngineIndexEndpoint(

index_endpoint_name=’my_index_endpoint_id’

)

  • Parameters

    • index_endpoint_name (str) – Required. A fully-qualified index endpoint resource name or a index ID. Example: “projects/123/locations/us-central1/index_endpoints/my_index_id” or “my_index_id” when project and location are initialized or passed.

    • project (str) – Optional. Project to retrieve index endpoint from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to retrieve index endpoint from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to retrieve this IndexEndpoint. Overrides credentials set in aiplatform.init.

client_class()

alias of google.cloud.aiplatform.utils.IndexEndpointClientWithOverride

classmethod create(display_name: str, network: Optional[str] = None, public_endpoint_enabled: bool = False, description: Optional[str] = None, labels: Optional[Dict[str, str]] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None, request_metadata: Optional[Sequence[Tuple[str, str]]] = (), sync: bool = True, enable_private_service_connect: bool = False, project_allowlist: Optional[Sequence[str]] = None, encryption_spec_key_name: Optional[str] = None, create_request_timeout: Optional[float] = None)

Creates a MatchingEngineIndexEndpoint resource.

Example Usage:

my_index_endpoint = aiplatform.IndexEndpoint.create(

display_name=’my_endpoint’,

)

  • Parameters

    • display_name (str) – Required. The display name of the IndexEndpoint. The name can be up to 128 characters long and can be consist of any UTF-8 characters.

    • network (str) – Optional. The full name of the Google Compute Engine network to which the IndexEndpoint should be peered.

      Private services access must already be configured for the network. If left unspecified, the network set with aiplatform.init will be used.

      Format: projects/{project}/global/networks/{network}. Where {project} is a project number, as in ‘12345’, and {network} is network name.

    • public_endpoint_enabled (bool) – Optional. If true, the deployed index will be accessible through public endpoint.

    • description (str) – Optional. The description of the IndexEndpoint.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The labels with user-defined metadata to organize your IndexEndpoint. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information on and examples of labels. No more than 64 user labels can be associated with one IndexEndpoint (System labels are excluded).” System reserved label keys are prefixed with “aiplatform.googleapis.com/” and are immutable.

    • project (str) – Optional. Project to create IndexEndpoint in. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to create IndexEndpoint in. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to create IndexEndpoints. Overrides credentials set in aiplatform.init.

    • request_metadata (Sequence[Tuple[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*]) – Optional. Strings which should be sent along with the request as metadata.

    • sync (bool) – Optional. Whether to execute this creation synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

    • enable_private_service_connect (bool) – If true, expose the index endpoint via private service connect.

    • project_allowlist (Sequence[str]) – Optional. List of projects from which the forwarding rule will target the service attachment.

    • encryption_spec_key_name (str) – Optional. The Cloud KMS resource identifier of the customer managed encryption key used to protect the index endpoint. Has the form: projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key. The key needs to be in the same region as where the compute resource is created.

      If set, this index endpoint and all sub-resources of this index endpoint will be secured by this key. The key needs to be in the same region as where the index endpoint is created.

    • create_request_timeout (float) – Optional. The timeout for the request in seconds.

  • Returns

    MatchingEngineIndexEndpoint - IndexEndpoint resource object

  • Raises

    ValueError – A network must be instantiated when creating a IndexEndpoint.

property create_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was created.

delete(force: bool = False, sync: bool = True)

Deletes this MatchingEngineIndexEndpoint resource. If force is set to True, all indexes on this endpoint will be undeployed prior to deletion.

  • Parameters

    • force (bool) – Required. If force is set to True, all deployed indexes on this endpoint will be undeployed first. Default is False.

    • sync (bool) – Whether to execute this method synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

  • Raises

    FailedPrecondition – If indexes are deployed on this MatchingEngineIndexEndpoint and force = False.

deploy_index(index: google.cloud.aiplatform.matching_engine.matching_engine_index.MatchingEngineIndex, deployed_index_id: str, display_name: Optional[str] = None, machine_type: Optional[str] = None, min_replica_count: Optional[int] = None, max_replica_count: Optional[int] = None, enable_access_logging: Optional[bool] = None, reserved_ip_ranges: Optional[Sequence[str]] = None, deployment_group: Optional[str] = None, auth_config_audiences: Optional[Sequence[str]] = None, auth_config_allowed_issuers: Optional[Sequence[str]] = None, request_metadata: Optional[Sequence[Tuple[str, str]]] = (), deploy_request_timeout: Optional[float] = None)

Deploys an existing index resource to this endpoint resource.

  • Parameters

    • index (MatchingEngineIndex) – Required. The Index this is the deployment of. We may refer to this Index as the DeployedIndex’s “original” Index.

    • deployed_index_id (str) – Required. The user specified ID of the DeployedIndex. The ID can be up to 128 characters long and must start with a letter and only contain letters, numbers, and underscores. The ID must be unique within the project it is created in.

    • display_name (str) – The display name of the DeployedIndex. If not provided upon creation, the Index’s display_name is used.

    • machine_type (str) – Optional. The type of machine. Not specifying machine type will result in model to be deployed with automatic resources.

    • min_replica_count (int) – Optional. The minimum number of machine replicas this deployed model will be always deployed on. If traffic against it increases, it may dynamically be deployed onto more replicas, and as traffic decreases, some of these extra replicas may be freed.

      If this value is not provided, the value of 2 will be used.

    • max_replica_count (int) – Optional. The maximum number of replicas this deployed model may be deployed on when the traffic against it increases. If requested value is too large, the deployment will error, but if deployment succeeds then the ability to scale the model to that many replicas is guaranteed (barring service outages). If traffic against the deployed model increases beyond what its replicas at maximum may handle, a portion of the traffic will be dropped. If this value is not provided, the larger value of min_replica_count or 2 will be used. If value provided is smaller than min_replica_count, it will automatically be increased to be min_replica_count.

    • enable_access_logging (bool) – Optional. If true, private endpoint’s access logs are sent to StackDriver Logging. These logs are like standard server access logs, containing information like timestamp and latency for each MatchRequest. Note that Stackdriver logs may incur a cost, especially if the deployed index receives a high queries per second rate (QPS). Estimate your costs before enabling this option.

    • reserved_ip_ranges (Sequence[str]) – Optional. A list of reserved ip ranges under the VPC network that can be used for this DeployedIndex. If set, we will deploy the index within the provided ip ranges. Otherwise, the index might be deployed to any ip ranges under the provided VPC network.

      The value sohuld be the name of the address (https://cloud.google.com/compute/docs/reference/rest/v1/addresses) Example: ‘vertex-ai-ip-range’.

    • deployment_group (str) – Optional. The deployment group can be no longer than 64 characters (eg: ‘test’, ‘prod’). If not set, we will use the ‘default’ deployment group.

      Creating deployment_groups with reserved_ip_ranges is a recommended practice when the peered network has multiple peering ranges. This creates your deployments from predictable IP spaces for easier traffic administration. Also, one deployment_group (except ‘default’) can only be used with the same reserved_ip_ranges which means if the deployment_group has been used with reserved_ip_ranges: [a, b, c], using it with [a, b] or [d, e] is disallowed.

      Note: we only support up to 5 deployment groups(not including ‘default’).

    • auth_config_audiences (Sequence[str]) – The list of JWT audiences. that are allowed to access. A JWT containing any of these audiences will be accepted.

      auth_config_audiences and auth_config_allowed_issuers must be passed together.

    • auth_config_allowed_issuers (Sequence[str]) – A list of allowed JWT issuers. Each entry must be a valid Google service account, in the following format:

      service-account-name@project-id.iam.gserviceaccount.com

      auth_config_audiences and auth_config_allowed_issuers must be passed together.

    • request_metadata (Sequence[Tuple[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*]) – Optional. Strings which should be sent along with the request as metadata.

    • deploy_request_timeout (float) – Optional. The timeout for the request in seconds.

  • Returns

    MatchingEngineIndexEndpoint - IndexEndpoint resource object

property deployed_indexes(: List[[google.cloud.aiplatform_v1.types.index_endpoint.DeployedIndex](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.DeployedIndex)_ )

Returns a list of deployed indexes on this endpoint.

  • Returns

    List[gca_matching_engine_index_endpoint.DeployedIndex] - Deployed indexes

property description(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Description of the index endpoint.

property display_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Display name of this resource.

property encryption_spec(: Optional[[google.cloud.aiplatform_v1.types.encryption_spec.EncryptionSpec](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.EncryptionSpec)_ )

Customer-managed encryption key options for this Vertex AI resource.

If this is set, then all resources created by this Vertex AI resource will be encrypted with the provided encryption key.

find_neighbors(*, deployed_index_id: str, queries: Optional[List[List[float]]] = None, num_neighbors: int = 10, filter: Optional[List[google.cloud.aiplatform.matching_engine.matching_engine_index_endpoint.Namespace]] = None, per_crowding_attribute_neighbor_count: Optional[int] = None, approx_num_neighbors: Optional[int] = None, fraction_leaf_nodes_to_search_override: Optional[float] = None, return_full_datapoint: bool = False, numeric_filter: Optional[List[google.cloud.aiplatform.matching_engine.matching_engine_index_endpoint.NumericNamespace]] = None, embedding_ids: Optional[List[str]] = None)

Retrieves nearest neighbors for the given embedding queries on the specified deployed index which is deployed to either public or private endpoint.

``

` Example usage:

my_index_endpoint = aiplatform.MatchingEngineIndexEndpoint(

index_endpoint_name=’projects/123/locations/us-central1/index_endpoint/my_index_endpoint_id’

) my_index_endpoint.find_neighbors(deployed_index_id=”deployed_index_id”, queries= [[1, 1]],)

``

` :param deployed_index_id: Required. The ID of the DeployedIndex to match the queries against. :type deployed_index_id: str :param queries: Required. A list of queries. Each query is a list of floats, representing a single embedding. :type queries: List[List[float]] :param num_neighbors: Required. The number of nearest neighbors to be retrieved from database for

each query.

  • Parameters

    • filter (List[Namespace]) – Optional. A list of Namespaces for filtering the matching results. For example, [Namespace(“color”, [“red”], []), Namespace(“shape”, [], [“squared”])] will match datapoints that satisfy “red color” but not include datapoints with “squared shape”. Please refer to https://cloud.google.com/vertex-ai/docs/matching-engine/filtering#json for more detail.

    • per_crowding_attribute_neighbor_count (int) – Optional. Crowding is a constraint on a neighbor list produced by nearest neighbor search requiring that no more than some value k’ of the k neighbors returned have the same value of crowding_attribute. It’s used for improving result diversity. This field is the maximum number of matches with the same crowding tag.

    • approx_num_neighbors (int) – Optional. The number of neighbors to find via approximate search before exact reordering is performed. If not set, the default value from scam config is used; if set, this value must be > 0.

    • fraction_leaf_nodes_to_search_override (float) – Optional. The fraction of the number of leaves to search, set at query time allows user to tune search performance. This value increase result in both search accuracy and latency increase. The value should be between 0.0 and 1.0.

    • return_full_datapoint (bool) – Optional. If set to true, the full datapoints (including all vector values and of the nearest neighbors are returned. Note that returning full datapoint will significantly increase the latency and cost of the query.

    • numeric_filter (list[NumericNamespace]) – Optional. A list of NumericNamespaces for filtering the matching results. For example: [NumericNamespace(name=”cost”, value_int=5, op=”GREATER”)] will match datapoints that its cost is greater than 5.

    • embedding_ids (str) – Optional. If queries is set, will use queries to do nearest neighbor search. If queries isn’t set, will first use embedding_ids to lookup embedding values from dataset, if embedding with embedding_ids exists in the dataset, do nearest neighbor search.

  • Returns

    List[List[MatchNeighbor]] - A list of nearest neighbors for each query.

property gca_resource(: [proto.message.Message](https://proto-plus-python.readthedocs.io/en/latest/reference/message.html#proto.message.Message )

The underlying resource proto representation.

property labels(: Dict[str, str )

User-defined labels containing metadata about this resource.

Read more about labels at https://goo.gl/xmQnxf

classmethod list(filter: Optional[str] = None, order_by: Optional[str] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None, parent: Optional[str] = None)

List all instances of this Vertex AI Resource.

Example Usage:

aiplatform.BatchPredictionJobs.list(

filter=’state=”JOB_STATE_SUCCEEDED” AND display_name=”my_job”’,

)

aiplatform.Model.list(order_by=”create_time desc, display_name”)

  • Parameters

    • filter (str) – Optional. An expression for filtering the results of the request. For field names both snake_case and camelCase are supported.

    • order_by (str) – Optional. A comma-separated list of fields to order by, sorted in ascending order. Use “desc” after a field name for descending. Supported fields: display_name, create_time, update_time

    • project (str) – Optional. Project to retrieve list from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to retrieve list from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to retrieve list. Overrides credentials set in aiplatform.init.

    • parent (str) – Optional. The parent resource name if any to retrieve list from.

  • Returns

    List[VertexAiResourceNoun] - A list of SDK resource objects

match(deployed_index_id: str, queries: Optional[List[List[float]]] = None, num_neighbors: int = 1, filter: Optional[List[google.cloud.aiplatform.matching_engine.matching_engine_index_endpoint.Namespace]] = None, per_crowding_attribute_num_neighbors: Optional[int] = None, approx_num_neighbors: Optional[int] = None, fraction_leaf_nodes_to_search_override: Optional[float] = None, low_level_batch_size: int = 0, numeric_filter: Optional[List[google.cloud.aiplatform.matching_engine.matching_engine_index_endpoint.NumericNamespace]] = None)

Retrieves nearest neighbors for the given embedding queries on the specified deployed index for private endpoint only.

  • Parameters

    • deployed_index_id (str) – Required. The ID of the DeployedIndex to match the queries against.

    • queries (List[List[float]]) – Optional. A list of queries. Each query is a list of floats, representing a single embedding.

    • num_neighbors (int) – Required. The number of nearest neighbors to be retrieved from database for each query.

    • filter (List[Namespace]) – Optional. A list of Namespaces for filtering the matching results. For example, [Namespace(“color”, [“red”], []), Namespace(“shape”, [], [“squared”])] will match datapoints that satisfy “red color” but not include datapoints with “squared shape”. Please refer to https://cloud.google.com/vertex-ai/docs/matching-engine/filtering#json for more detail.

    • per_crowding_attribute_num_neighbors (int) – Optional. Crowding is a constraint on a neighbor list produced by nearest neighbor search requiring that no more than some value k’ of the k neighbors returned have the same value of crowding_attribute. It’s used for improving result diversity. This field is the maximum number of matches with the same crowding tag.

    • approx_num_neighbors (int) – The number of neighbors to find via approximate search before exact reordering is performed. If not set, the default value from scam config is used; if set, this value must be > 0.

    • fraction_leaf_nodes_to_search_override (float) – Optional. The fraction of the number of leaves to search, set at query time allows user to tune search performance. This value increase result in both search accuracy and latency increase. The value should be between 0.0 and 1.0.

    • low_level_batch_size (int) – Optional. Selects the optimal batch size to use for low-level batching. Queries within each low level batch are executed sequentially while low level batches are executed in parallel. This field is optional, defaults to 0 if not set. A non-positive number disables low level batching, i.e. all queries are executed sequentially.

    • numeric_filter (Optional[list[NumericNamespace]]) – Optional. A list of NumericNamespaces for filtering the matching results. For example: [NumericNamespace(name=”cost”, value_int=5, op=”GREATER”)] will match datapoints that its cost is greater than 5.

  • Returns

    List[List[MatchNeighbor]] - A list of nearest neighbors for each query.

mutate_deployed_index(deployed_index_id: str, min_replica_count: int = 1, max_replica_count: int = 1, request_metadata: Optional[Sequence[Tuple[str, str]]] = (), mutate_request_timeout: Optional[float] = None)

Updates an existing deployed index under this endpoint resource.

  • Parameters

    • index_id (str) – Required. The ID of the MatchingEnginIndex associated with the DeployedIndex.

    • deployed_index_id (str) – Required. The user specified ID of the DeployedIndex. The ID can be up to 128 characters long and must start with a letter and only contain letters, numbers, and underscores. The ID must be unique within the project it is created in.

    • min_replica_count (int) – Optional. The minimum number of machine replicas this deployed model will be always deployed on. If traffic against it increases, it may dynamically be deployed onto more replicas, and as traffic decreases, some of these extra replicas may be freed.

    • max_replica_count (int) – Optional. The maximum number of replicas this deployed model may be deployed on when the traffic against it increases. If requested value is too large, the deployment will error, but if deployment succeeds then the ability to scale the model to that many replicas is guaranteed (barring service outages). If traffic against the deployed model increases beyond what its replicas at maximum may handle, a portion of the traffic will be dropped. If this value is not provided, the larger value of min_replica_count or 1 will be used. If value provided is smaller than min_replica_count, it will automatically be increased to be min_replica_count.

    • request_metadata (Sequence[Tuple[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*]) – Optional. Strings which should be sent along with the request as metadata.

    • timeout (float) – Optional. The timeout for the request in seconds.

property name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Name of this resource.

property private_service_access_network(: Optional[str )

“Private service access network.

property private_service_connect_ip_address(: Optional[str )

“Private service connect ip address.

property public_endpoint_domain_name(: Optional[str )

Public endpoint DNS name.

read_index_datapoints(*, deployed_index_id: str, ids: List[str] = [])

Reads the datapoints/vectors of the given IDs on the specified deployed index which is deployed to public or private endpoint.

``

` Example Usage:

my_index_endpoint = aiplatform.MatchingEngineIndexEndpoint(

index_endpoint_name=’projects/123/locations/us-central1/index_endpoint/my_index_id’

) my_index_endpoint.read_index_datapoints(deployed_index_id=”public_test1”, ids= [“606431”, “896688”],)

``

`

  • Parameters

    • deployed_index_id (str) – Required. The ID of the DeployedIndex to match the queries against.

    • ids (List[str]) – Required. IDs of the datapoints to be searched for.

  • Returns

    List[gca_index_v1beta1.IndexDatapoint] - A list of datapoints/vectors of the given IDs.

property resource_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Full qualified resource name.

to_dict()

Returns the resource proto as a dictionary.

undeploy_all(sync: bool = True)

Undeploys every index deployed to this MatchingEngineIndexEndpoint.

  • Parameters

    sync (bool) – Whether to execute this method synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

undeploy_index(deployed_index_id: str, request_metadata: Optional[Sequence[Tuple[str, str]]] = (), undeploy_request_timeout: Optional[float] = None)

Undeploy a deployed index endpoint resource.

  • Parameters

    • deployed_index_id (str) – Required. The ID of the DeployedIndex to be undeployed from the IndexEndpoint.

    • request_metadata (Sequence[Tuple[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*]) – Optional. Strings which should be sent along with the request as metadata.

    • undeploy_request_timeout (float) – Optional. The timeout for the request in seconds.

  • Returns

    MatchingEngineIndexEndpoint - IndexEndpoint resource object

update(display_name: str, description: Optional[str] = None, labels: Optional[Dict[str, str]] = None, request_metadata: Optional[Sequence[Tuple[str, str]]] = (), update_request_timeout: Optional[float] = None)

Updates an existing index endpoint resource.

  • Parameters

    • display_name (str) – Required. The display name of the IndexEndpoint. The name can be up to 128 characters long and can be consist of any UTF-8 characters.

    • description (str) – Optional. The description of the IndexEndpoint.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The labels with user-defined metadata to organize your Indexs. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information on and examples of labels. No more than 64 user labels can be associated with one IndexEndpoint (System labels are excluded).” System reserved label keys are prefixed with “aiplatform.googleapis.com/” and are immutable.

    • request_metadata (Sequence[Tuple[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*]) – Optional. Strings which should be sent along with the request as metadata.

    • update_request_timeout (float) – Optional. The timeout for the request in seconds.

  • Returns

    MatchingEngineIndexEndpoint - The updated index endpoint resource object.

property update_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was last updated.

wait()

Helper method that blocks until all futures are complete.

class google.cloud.aiplatform.Model(model_name: str, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None, version: Optional[str] = None)

Bases: google.cloud.aiplatform.base.VertexAiResourceNounWithFutureManager, google.cloud.aiplatform.base.PreviewMixin

Retrieves the model resource and instantiates its representation.

  • Parameters

    • model_name (str) – Required. A fully-qualified model resource name or model ID. Example: “projects/123/locations/us-central1/models/456” or “456” when project and location are initialized or passed. May optionally contain a version ID or version alias in {model_name}@{version} form. See version arg.

    • project (str) – Optional project to retrieve model from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional location to retrieve model from. If not set, location set in aiplatform.init will be used.

    • credentials – Optional[auth_credentials.Credentials]=None, Custom credentials to use to upload this model. If not set, credentials set in aiplatform.init will be used.

    • version (str) – Optional. Version ID or version alias. When set, the specified model version will be targeted unless overridden in method calls. When not set, the model with the “default” alias will be targeted unless overridden in method calls. No behavior change if only one version of a model exists.

  • Raises

    ValueError – If version is passed alongside a model_name referencing a different version.

batch_predict(job_display_name: Optional[str] = None, gcs_source: Optional[Union[str, Sequence[str]]] = None, bigquery_source: Optional[str] = None, instances_format: str = 'jsonl', gcs_destination_prefix: Optional[str] = None, bigquery_destination_prefix: Optional[str] = None, predictions_format: str = 'jsonl', model_parameters: Optional[Dict] = None, machine_type: Optional[str] = None, accelerator_type: Optional[str] = None, accelerator_count: Optional[int] = None, starting_replica_count: Optional[int] = None, max_replica_count: Optional[int] = None, generate_explanation: Optional[bool] = False, explanation_metadata: Optional[google.cloud.aiplatform_v1.types.explanation_metadata.ExplanationMetadata] = None, explanation_parameters: Optional[google.cloud.aiplatform_v1.types.explanation.ExplanationParameters] = None, labels: Optional[Dict[str, str]] = None, credentials: Optional[google.auth.credentials.Credentials] = None, encryption_spec_key_name: Optional[str] = None, sync: bool = True, create_request_timeout: Optional[float] = None, batch_size: Optional[int] = None, service_account: Optional[str] = None)

Creates a batch prediction job using this Model and outputs prediction results to the provided destination prefix in the specified predictions_format. One source and one destination prefix are required.

Example usage:

my_model.batch_predict(

    job_display_name=”prediction-123”,
    gcs_source=”gs://example-bucket/instances.csv”,
    instances_format=”csv”,
    bigquery_destination_prefix=”projectId.bqDatasetId.bqTableId”

)

  • Parameters

    • job_display_name (str) – Optional. The user-defined name of the BatchPredictionJob. The name can be up to 128 characters long and can be consist of any UTF-8 characters.

    • gcs_source – Optional[Sequence[str]] = None Google Cloud Storage URI(-s) to your instances to run batch prediction on. They must match instances_format.

    • bigquery_source – Optional[str] = None BigQuery URI to a table, up to 2000 characters long. For example: bq://projectId.bqDatasetId.bqTableId

    • instances_format – str = “jsonl” The format in which instances are provided. Must be one of the formats listed in Model.supported_input_storage_formats. Default is “jsonl” when using gcs_source. If a bigquery_source is provided, this is overridden to “bigquery”.

    • gcs_destination_prefix – Optional[str] = None The Google Cloud Storage location of the directory where the output is to be written to. In the given directory a new directory is created. Its name is prediction-<model-display-name>-<job-create-time>, where timestamp is in YYYY-MM-DDThh:mm:ss.sssZ ISO-8601 format. Inside of it files predictions_0001.<extension>, predictions_0002.<extension>, …, predictions_N.<extension> are created where <extension> depends on chosen predictions_format, and N may equal 0001 and depends on the total number of successfully predicted instances. If the Model has both instance and prediction schemata defined then each such file contains predictions as per the predictions_format. If prediction for any instance failed (partially or completely), then an additional errors_0001.<extension>, errors_0002.<extension>,…, errors_N.<extension> files are created (N depends on total number of failed predictions). These files contain the failed instances, as per their schema, followed by an additional error field which as value has \google.rpc.Status<Status>\__ containing only code and message fields.

    • bigquery_destination_prefix – Optional[str] = None The BigQuery URI to a project or table, up to 2000 characters long. When only the project is specified, the Dataset and Table is created. When the full table reference is specified, the Dataset must exist and table must not exist. Accepted forms: bq://projectId or bq://projectId.bqDatasetId. If no Dataset is specified, a new one is created with the name prediction_<model-display-name>_<job-create-time> where the table name is made BigQuery-dataset-name compatible (for example, most special characters become underscores), and timestamp is in YYYY_MM_DDThh_mm_ss_sssZ “based on ISO-8601” format. In the dataset two tables will be created, predictions, and errors. If the Model has both instance and prediction schemata defined then the tables have columns as follows: The predictions table contains instances for which the prediction succeeded, it has columns as per a concatenation of the Model’s instance and prediction schemata. The errors table contains rows for which the prediction has failed, it has instance columns, as per the instance schema, followed by a single “errors” column, which as values has \google.rpc.Status<Status>\__ represented as a STRUCT, and containing only code and message.

    • predictions_format – str = “jsonl” Required. The format in which Vertex AI outputs the predictions, must be one of the formats specified in Model.supported_output_storage_formats. Default is “jsonl” when using gcs_destination_prefix. If a bigquery_destination_prefix is provided, this is overridden to “bigquery”.

    • model_parameters – Optional[Dict] = None Optional. The parameters that govern the predictions. The schema of the parameters may be specified via the Model’s parameters_schema_uri.

    • machine_type – Optional[str] = None Optional. The type of machine for running batch prediction on dedicated resources. Not specifying machine type will result in batch prediction job being run with automatic resources.

    • accelerator_type – Optional[str] = None Optional. The type of accelerator(s) that may be attached to the machine as per accelerator_count. Only used if machine_type is set.

    • accelerator_count – Optional[int] = None Optional. The number of accelerators to attach to the machine_type. Only used if machine_type is set.

    • starting_replica_count – Optional[int] = None The number of machine replicas used at the start of the batch operation. If not set, Vertex AI decides starting number, not greater than max_replica_count. Only used if machine_type is set.

    • max_replica_count – Optional[int] = None The maximum number of machine replicas the batch operation may be scaled to. Only used if machine_type is set. Default is 10.

    • generate_explanation (bool) – Optional. Generate explanation along with the batch prediction results. This will cause the batch prediction output to include explanations based on the prediction_format:

      * bigquery: output includes a column named explanation. The value

is a struct that conforms to the [aiplatform.gapic.Explanation] object.

>     * jsonl: The JSON objects on each line include an additional entry

>     keyed explanation. The value of the entry is a JSON object that
>     conforms to the [aiplatform.gapic.Explanation] object.


>     * csv: Generating explanations for CSV format is not supported.



* **explanation_metadata** (*aiplatform.explain.ExplanationMetadata*) – Optional. Explanation metadata configuration for this BatchPredictionJob.
Can be specified only if generate_explanation is set to True.

This value overrides the value of Model.explanation_metadata.
All fields of explanation_metadata are optional in the request. If
a field of the explanation_metadata object is not populated, the
corresponding field of the Model.explanation_metadata object is inherited.
For more details, see Ref docs <http://tinyurl.com/1igh60kt>



* **explanation_parameters** (*aiplatform.explain.ExplanationParameters*) – Optional. Parameters to configure explaining for Model’s predictions.
Can be specified only if generate_explanation is set to True.

This value overrides the value of Model.explanation_parameters.
All fields of explanation_parameters are optional in the request. If
a field of the explanation_parameters object is not populated, the
corresponding field of the Model.explanation_parameters object is inherited.
For more details, see Ref docs <http://tinyurl.com/1an4zake>



* **labels** – Optional[Dict[str, str]] = None
Optional. The labels with user-defined metadata to organize your
BatchPredictionJobs. Label keys and values can be no longer than
64 characters (Unicode codepoints), can only contain lowercase
letters, numeric characters, underscores and dashes.
International characters are allowed. See [https://goo.gl/xmQnxf](https://goo.gl/xmQnxf)
for more information and examples of labels.


* **credentials** – Optional[auth_credentials.Credentials] = None
Optional. Custom credentials to use to create this batch prediction
job. Overrides credentials set in aiplatform.init.


* **encryption_spec_key_name** (*Optional**[*[*str*](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)*]*) – Optional. The Cloud KMS resource identifier of the customer
managed encryption key used to protect the model. Has the
form:
`projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key`.
The key needs to be in the same region as where the compute
resource is created.

If set, this Model and all sub-resources of this Model will be secured by this key.

Overrides encryption_spec_key_name set in aiplatform.init.



* **create_request_timeout** ([*float*](https://python.readthedocs.io/en/latest/library/functions.html#float)) – Optional. The timeout for the create request in seconds.


* **batch_size** ([*int*](https://python.readthedocs.io/en/latest/library/functions.html#int)) – Optional. The number of the records (e.g. instances) of the operation given in each batch
to a machine replica. Machine type, and size of a single record should be considered
when setting this parameter, higher value speeds up the batch operation’s execution,
but too high value will result in a whole batch not fitting in a machine’s memory,
and the whole operation will fail.
The default value is 64.


* **service_account** ([*str*](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)) – Optional. Specifies the service account for workload run-as account.
Users submitting jobs must have act-as permission on this run-as account.

  • Returns

    Instantiated representation of the created batch prediction job.

  • Return type

    job (jobs.BatchPredictionJob)

client_class()

alias of google.cloud.aiplatform.utils.ModelClientWithOverride

property container_spec(: Optional[[google.cloud.aiplatform_v1.types.model.ModelContainerSpec](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.ModelContainerSpec)_ )

The specification of the container that is to be used when deploying this Model. Not present for AutoML Models.

copy(destination_location: str, destination_model_id: Optional[str] = None, destination_parent_model: Optional[str] = None, encryption_spec_key_name: Optional[str] = None, copy_request_timeout: Optional[float] = None)

Copys a model and returns a Model representing the copied Model resource. This method is a blocking call.

Example usage:

copied_model = my_model.copy(

    destination_location=”us-central1”

)

  • Parameters

    • destination_location (str) – The destination location to copy the model to.

    • destination_model_id (str) – Optional. The ID to use for the copied Model, which will become the final component of the model resource name. This value may be up to 63 characters, and valid characters are [a-z0-9_-]. The first character cannot be a number or hyphen.

      Only set this field when copying as a new model. If this field is not set, a numeric model id will be generated.

    • destination_parent_model (str) – Optional. The resource name or model ID of an existing model that the newly-copied model will be a version of.

      Only set this field when copying as a new version of an existing model.

    • encryption_spec_key_name (Optional[str]) – Optional. The Cloud KMS resource identifier of the customer managed encryption key used to protect the model. Has the form: projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key. The key needs to be in the same region as where the compute resource is created.

      If set, this Model and all sub-resources of this Model will be secured by this key.

      Overrides encryption_spec_key_name set in aiplatform.init.

    • copy_request_timeout (float) – Optional. The timeout for the copy request in seconds.

  • Returns

    Instantiated representation of the copied model resource.

  • Return type

    model (aiplatform.Model)

  • Raises

    ValueError – If both destination_model_id and destination_parent_model are set.

property create_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was created.

delete(sync: bool = True)

Deletes this Vertex AI resource. WARNING: This deletion is permanent.

  • Parameters

    sync (bool) – Whether to execute this deletion synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

deploy(endpoint: Optional[Union[google.cloud.aiplatform.models.Endpoint, google.cloud.aiplatform.models.PrivateEndpoint]] = None, deployed_model_display_name: Optional[str] = None, traffic_percentage: Optional[int] = 0, traffic_split: Optional[Dict[str, int]] = None, machine_type: Optional[str] = None, min_replica_count: int = 1, max_replica_count: int = 1, accelerator_type: Optional[str] = None, accelerator_count: Optional[int] = None, tpu_topology: Optional[str] = None, service_account: Optional[str] = None, explanation_metadata: Optional[google.cloud.aiplatform_v1.types.explanation_metadata.ExplanationMetadata] = None, explanation_parameters: Optional[google.cloud.aiplatform_v1.types.explanation.ExplanationParameters] = None, metadata: Optional[Sequence[Tuple[str, str]]] = (), encryption_spec_key_name: Optional[str] = None, network: Optional[str] = None, sync=True, deploy_request_timeout: Optional[float] = None, autoscaling_target_cpu_utilization: Optional[int] = None, autoscaling_target_accelerator_duty_cycle: Optional[int] = None, enable_access_logging=False, disable_container_logging: bool = False)

Deploys model to endpoint. Endpoint will be created if unspecified.

  • Parameters

    • endpoint (Union[Endpoint, **PrivateEndpoint]) – Optional. Public or private Endpoint to deploy model to. If not specified, endpoint display name will be model display name+’_endpoint’.

    • deployed_model_display_name (str) – Optional. The display name of the DeployedModel. If not provided upon creation, the Model’s display_name is used.

    • traffic_percentage (int) – Optional. Desired traffic to newly deployed model. Defaults to 0 if there are pre-existing deployed models. Defaults to 100 if there are no pre-existing deployed models. Negative values should not be provided. Traffic of previously deployed models at the endpoint will be scaled down to accommodate new deployed model’s traffic. Should not be provided if traffic_split is provided.

    • traffic_split (Dict[str, *[int](https://python.readthedocs.io/en/latest/library/functions.html#int)]*) – Optional. A map from a DeployedModel’s ID to the percentage of this Endpoint’s traffic that should be forwarded to that DeployedModel. If a DeployedModel’s ID is not listed in this map, then it receives no traffic. The traffic percentage values must add up to 100, or map must be empty if the Endpoint is to not accept any traffic at the moment. Key for model being deployed is “0”. Should not be provided if traffic_percentage is provided.

    • machine_type (str) – Optional. The type of machine. Not specifying machine type will result in model to be deployed with automatic resources.

    • min_replica_count (int) – Optional. The minimum number of machine replicas this deployed model will be always deployed on. If traffic against it increases, it may dynamically be deployed onto more replicas, and as traffic decreases, some of these extra replicas may be freed.

    • max_replica_count (int) – Optional. The maximum number of replicas this deployed model may be deployed on when the traffic against it increases. If requested value is too large, the deployment will error, but if deployment succeeds then the ability to scale the model to that many replicas is guaranteed (barring service outages). If traffic against the deployed model increases beyond what its replicas at maximum may handle, a portion of the traffic will be dropped. If this value is not provided, the smaller value of min_replica_count or 1 will be used.

    • accelerator_type (str) – Optional. Hardware accelerator type. Must also set accelerator_count if used. One of ACCELERATOR_TYPE_UNSPECIFIED, NVIDIA_TESLA_K80, NVIDIA_TESLA_P100, NVIDIA_TESLA_V100, NVIDIA_TESLA_P4, NVIDIA_TESLA_T4

    • accelerator_count (int) – Optional. The number of accelerators to attach to a worker replica.

    • tpu_topology (str) – Optional. The TPU topology to use for the DeployedModel. Requireid for CloudTPU multihost deployments.

    • service_account (str) – The service account that the DeployedModel’s container runs as. Specify the email address of the service account. If this service account is not specified, the container runs as a service account that doesn’t have access to the resource project. Users deploying the Model must have the iam.serviceAccounts.actAs permission on this service account.

    • explanation_metadata (aiplatform.explain.ExplanationMetadata) – Optional. Metadata describing the Model’s input and output for explanation. explanation_metadata is optional while explanation_parameters must be specified when used. For more details, see Ref docs http://tinyurl.com/1igh60kt

    • explanation_parameters (aiplatform.explain.ExplanationParameters) – Optional. Parameters to configure explaining for Model’s predictions. For more details, see Ref docs http://tinyurl.com/1an4zake

    • metadata (Sequence[Tuple[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*]) – Optional. Strings which should be sent along with the request as metadata.

    • encryption_spec_key_name (Optional[str]) – Optional. The Cloud KMS resource identifier of the customer managed encryption key used to protect the model. Has the form: projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key. The key needs to be in the same region as where the compute resource is created.

      If set, this Endpoint and all sub-resources of this Endpoint will be secured by this key.

      Overrides encryption_spec_key_name set in aiplatform.init.

    • network (str) – Optional. The full name of the Compute Engine network to which the Endpoint, if created, will be peered to. E.g. “projects/12345/global/networks/myVPC”. Private services access must already be configured for the network. If set or aiplatform.init(network=…) has been set, a PrivateEndpoint will be created. If left unspecified, an Endpoint will be created. Read more about PrivateEndpoints in the documentation.

    • sync (bool) – Whether to execute this method synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

    • deploy_request_timeout (float) – Optional. The timeout for the deploy request in seconds.

    • autoscaling_target_cpu_utilization (int) – Optional. Target CPU Utilization to use for Autoscaling Replicas. A default value of 60 will be used if not specified.

    • autoscaling_target_accelerator_duty_cycle (int) – Optional. Target Accelerator Duty Cycle. Must also set accelerator_type and accelerator_count if specified. A default value of 60 will be used if not specified.

    • enable_access_logging (bool) – Whether to enable endpoint access logging. Defaults to False.

    • disable_container_logging (bool) – If True, container logs from the deployed model will not be written to Cloud Logging. Defaults to False.

  • Returns

    Endpoint with the deployed model.

  • Return type

    endpoint (Union[Endpoint, PrivateEndpoint])

  • Raises

    ValueError – If traffic_split is set for PrivateEndpoint.

property description(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Description of the model.

property display_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Display name of this resource.

property encryption_spec(: Optional[[google.cloud.aiplatform_v1.types.encryption_spec.EncryptionSpec](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.EncryptionSpec)_ )

Customer-managed encryption key options for this Vertex AI resource.

If this is set, then all resources created by this Vertex AI resource will be encrypted with the provided encryption key.

evaluate(prediction_type: str, target_field_name: str, gcs_source_uris: Optional[List[str]] = None, bigquery_source_uri: Optional[str] = None, bigquery_destination_output_uri: Optional[str] = None, class_labels: Optional[List[str]] = None, prediction_label_column: Optional[str] = None, prediction_score_column: Optional[str] = None, staging_bucket: Optional[str] = None, service_account: Optional[str] = None, generate_feature_attributions: bool = False, evaluation_pipeline_display_name: Optional[str] = None, evaluation_metrics_display_name: Optional[str] = None, network: Optional[str] = None, encryption_spec_key_name: Optional[str] = None, experiment: Optional[Union[google.cloud.aiplatform.metadata.experiment_resources.Experiment, str]] = None)

Creates a model evaluation job running on Vertex Pipelines and returns the resulting ModelEvaluationJob resource.

Example usage:

``

` my_model = Model(

model_name=”projects/123/locations/us-central1/models/456”

) my_evaluation_job = my_model.evaluate(

prediction_type=”classification”, target_field_name=”type”, data_source_uris=[“gs://sdk-model-eval/my-prediction-data.csv”], staging_bucket=”gs://my-staging-bucket/eval_pipeline_root”,

) my_evaluation_job.wait() my_evaluation = my_evaluation_job.get_model_evaluation() my_evaluation.metrics

``

`

  • Parameters

    • prediction_type (str) – Required. The problem type being addressed by this evaluation run. ‘classification’ and ‘regression’ are the currently supported problem types.

    • target_field_name (str) – Required. The column name of the field containing the label for this prediction task.

    • gcs_source_uris (List[str]) – Optional. A list of Cloud Storage data files containing the ground truth data to use for this evaluation job. These files should contain your model’s prediction column. Currently only Google Cloud Storage urls are supported, for example: “gs://path/to/your/data.csv”. The provided data files must be either CSV or JSONL. One of gcs_source_uris or bigquery_source_uri is required.

    • bigquery_source_uri (str) – Optional. A bigquery table URI containing the ground truth data to use for this evaluation job. This uri should be in the format ‘bq://my-project-id.dataset.table’. One of gcs_source_uris or bigquery_source_uri is required.

    • bigquery_destination_output_uri (str) – Optional. A bigquery table URI where the Batch Prediction job associated with your Model Evaluation will write prediction output. This can be a BigQuery URI to a project (‘bq://my-project’), a dataset (‘bq://my-project.my-dataset’), or a table (‘bq://my-project.my-dataset.my-table’). Required if bigquery_source_uri is provided.

    • class_labels (List[str]) – Optional. For custom (non-AutoML) classification models, a list of possible class names, in the same order that predictions are generated. This argument is required when prediction_type is ‘classification’. For example, in a classification model with 3 possible classes that are outputted in the format: [0.97, 0.02, 0.01] with the class names “cat”, “dog”, and “fish”, the value of class_labels should be [“cat”, “dog”, “fish”] where the class “cat” corresponds with 0.97 in the example above.

    • prediction_label_column (str) – Optional. The column name of the field containing classes the model is scoring. Formatted to be able to find nested columns, delimeted by .. If not set, defaulted to prediction.classes for classification.

    • prediction_score_column (str) – Optional. The column name of the field containing batch prediction scores. Formatted to be able to find nested columns, delimeted by .. If not set, defaulted to prediction.scores for a classification problem_type, prediction.value for a regression problem_type.

    • staging_bucket (str) – Optional. The GCS directory to use for staging files from this evaluation job. Defaults to the value set in aiplatform.init(staging_bucket=…) if not provided. Required if staging_bucket is not set in aiplatform.init().

    • service_account (str) – Specifies the service account for workload run-as account for this Model Evaluation PipelineJob. Users submitting jobs must have act-as permission on this run-as account. The service account running this Model Evaluation job needs the following permissions: Dataflow Worker, Storage Admin, Vertex AI Administrator, and Vertex AI Service Agent.

    • generate_feature_attributions (boolean) – Optional. Whether the model evaluation job should generate feature attributions. Defaults to False if not specified.

    • evaluation_pipeline_display_name (str) – Optional. The display name of your model evaluation job. This is the display name that will be applied to the Vertex Pipeline run for your evaluation job. If not set, a display name will be generated automatically.

    • evaluation_metrics_display_name (str) – Optional. The display name of the model evaluation resource uploaded to Vertex from your Model Evaluation pipeline.

    • network (str) – The full name of the Compute Engine network to which the job should be peered. For example, projects/12345/global/networks/myVPC. Private services access must already be configured for the network. If left unspecified, the job is not peered with any network.

    • encryption_spec_key_name (str) – Optional. The Cloud KMS resource identifier of the customer managed encryption key used to protect the job. Has the form: projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key. The key needs to be in the same region as where the compute resource is created. If this is set, then all resources created by the PipelineJob for this Model Evaluation will be encrypted with the provided encryption key. If not specified, encryption_spec of original PipelineJob will be used.

    • experiment (Union[str, **experiments_resource.Experiment]) – Optional. The Vertex AI experiment name or instance to associate to the PipelineJob executing this model evaluation job. Metrics produced by the PipelineJob as system.Metric Artifacts will be associated as metrics to the provided experiment, and parameters from this PipelineJob will be associated as parameters to the provided experiment.

  • Returns

    Instantiated representation of the _ModelEvaluationJob.

  • Return type

    model_evaluation.ModelEvaluationJob

  • Raises

    ValueError – If staging_bucket was not set in aiplatform.init() and staging_bucket was not provided. If the provided prediction_type is not valid. If the provided data_source_uris don’t start with ‘gs://’.

export_model(export_format_id: str, artifact_destination: Optional[str] = None, image_destination: Optional[str] = None, sync: bool = True)

Exports a trained, exportable Model to a location specified by the user. A Model is considered to be exportable if it has at least one supported_export_formats. Either artifact_destination or image_destination must be provided.

Example Usage:

my_model.export(

    export_format_id=”tf-saved-model”,
    artifact_destination=”gs://my-bucket/models/”

)

or

my_model.export(

    export_format_id=”custom-model”,
    image_destination=”us-central1-docker.pkg.dev/projectId/repo/image”

)

  • Parameters

    • export_format_id (str) – Required. The ID of the format in which the Model must be exported. The list of export formats that this Model supports can be found by calling Model.supported_export_formats.

    • artifact_destination (str) – The Cloud Storage location where the Model artifact is to be written to. Under the directory given as the destination a new one with name “model-export-<model-display-name>-<timestamp-of-export-call>”, where timestamp is in YYYY-MM-DDThh:mm:ss.sssZ ISO-8601 format, will be created. Inside, the Model and any of its supporting files will be written.

      This field should only be set when, in [Model.supported_export_formats], the value for the key given in export_format_id contains ARTIFACT.

    • image_destination (str) – The Google Container Registry or Artifact Registry URI where the Model container image will be copied to. Accepted forms:

      • Google Container Registry path. For example:

      gcr.io/projectId/imageName:tag.

      • Artifact Registry path. For example:

      us-central1-docker.pkg.dev/projectId/repoName/imageName:tag.

      This field should only be set when, in [Model.supported_export_formats], the value for the key given in export_format_id contains IMAGE.

    • sync (bool) – Whether to execute this export synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

  • Returns

    Details of the completed export with output destination paths to the artifacts or container image.

  • Return type

    output_info (Dict[str, str])

  • Raises

    • ValueError – If model does not support exporting.

    • ValueError – If invalid arguments or export formats are provided.

property gca_resource(: [proto.message.Message](https://proto-plus-python.readthedocs.io/en/latest/reference/message.html#proto.message.Message )

The underlying resource proto representation.

get_model_evaluation(evaluation_id: Optional[str] = None)

Returns a ModelEvaluation resource and instantiates its representation. If no evaluation_id is passed, it will return the first evaluation associated with this model. If the aiplatform.Model resource was instantiated with a version, this will return a Model Evaluation from that version. If no version was specified when instantiating the Model resource, this will return an Evaluation from the default version.

Example usage:

my_model = Model(

    model_name=”projects/123/locations/us-central1/models/456”

)

my_evaluation = my_model.get_model_evaluation(

    evaluation_id=”789”

)

# If no arguments are passed, this method returns the first evaluation for the model
my_evaluation = my_model.get_model_evaluation()

  • Parameters

    evaluation_id (str) – Optional. The ID of the model evaluation to retrieve.

  • Returns

    Instantiated representation of the ModelEvaluation resource.

  • Return type

    model_evaluation.ModelEvaluation

property labels(: Dict[str, str )

User-defined labels containing metadata about this resource.

Read more about labels at https://goo.gl/xmQnxf

classmethod list(filter: Optional[str] = None, order_by: Optional[str] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

List all Model resource instances.

Example Usage:

aiplatform.Model.list(

    filter=’labels.my_label=”my_label_value” AND display_name=”my_model”’,

)

  • Parameters

    • filter (str) – Optional. An expression for filtering the results of the request. For field names both snake_case and camelCase are supported.

    • order_by (str) – Optional. A comma-separated list of fields to order by, sorted in ascending order. Use “desc” after a field name for descending. Supported fields: display_name, create_time, update_time

    • project (str) – Optional. Project to retrieve list from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to retrieve list from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to retrieve list. Overrides credentials set in aiplatform.init.

  • Returns

    A list of Model resource objects

  • Return type

    List[models.Model]

list_model_evaluations()

List all Model Evaluation resources associated with this model. If this Model resource was instantiated with a version, the Model Evaluation resources for that version will be returned. If no version was provided when the Model resource was instantiated, Model Evaluation resources will be returned for the default version.

Example Usage:

my_model = Model(

    model_name=”projects/123/locations/us-central1/models/456@1”

)

my_evaluations = my_model.list_model_evaluations()

  • Returns

    List of ModelEvaluation resources for the model.

  • Return type

    List[model_evaluation.ModelEvaluation]

property name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Name of this resource.

property predict_schemata(: Optional[[google.cloud.aiplatform_v1.types.model.PredictSchemata](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.PredictSchemata)_ )

The schemata that describe formats of the Model’s predictions and explanations, if available.

property preview()

Return a Model instance with preview features enabled.

property resource_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Full qualified resource name, without any version ID.

property supported_deployment_resources_types(: List[[google.cloud.aiplatform_v1.types.model.Model.DeploymentResourcesType](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.Model.DeploymentResourcesType)_ )

List of deployment resource types accepted for this Model.

When this Model is deployed, its prediction resources are described by the prediction_resources field of the objects returned by Endpoint.list_models(). Because not all Models support all resource configuration types, the configuration types this Model supports are listed here.

If no configuration types are listed, the Model cannot be deployed to an Endpoint and does not support online predictions (Endpoint.predict() or Endpoint.explain()). Such a Model can serve predictions by using a BatchPredictionJob, if it has at least one entry each in Model.supported_input_storage_formats and Model.supported_output_storage_formats.

property supported_export_formats(: Dict[str, List[[google.cloud.aiplatform_v1.types.model.Model.ExportFormat.ExportableContent](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.Model.ExportFormat.ExportableContent)]_ )

The formats and content types in which this Model may be exported. If empty, this Model is not available for export.

For example, if this model can be exported as a Tensorflow SavedModel and have the artifacts written to Cloud Storage, the expected value would be:

{‘tf-saved-model’: [<ExportableContent.ARTIFACT: 1>]}

property supported_input_storage_formats(: List[str )

The formats this Model supports in the input_config field of a BatchPredictionJob. If Model.predict_schemata.instance_schema_uri exists, the instances should be given as per that schema.

Read the docs for more on batch prediction formats

If this Model doesn’t support any of these formats it means it cannot be used with a BatchPredictionJob. However, if it has supported_deployment_resources_types, it could serve online predictions by using Endpoint.predict() or Endpoint.explain().

property supported_output_storage_formats(: List[str )

The formats this Model supports in the output_config field of a BatchPredictionJob.

If both Model.predict_schemata.instance_schema_uri and Model.predict_schemata.prediction_schema_uri exist, the predictions are returned together with their instances. In other words, the prediction has the original instance data first, followed by the actual prediction content (as per the schema).

Read the docs for more on batch prediction formats

If this Model doesn’t support any of these formats it means it cannot be used with a BatchPredictionJob. However, if it has supported_deployment_resources_types, it could serve online predictions by using Endpoint.predict() or Endpoint.explain().

to_dict()

Returns the resource proto as a dictionary.

property training_job(: Optional[google.cloud.aiplatform.training_jobs._TrainingJob )

The TrainingJob that uploaded this Model, if any.

  • Raises

    api_core.exceptions.NotFound – If the Model’s training job resource cannot be found on the Vertex service.

update(display_name: Optional[str] = None, description: Optional[str] = None, labels: Optional[Dict[str, str]] = None)

Updates a model.

Example usage:

my_model = my_model.update(

    display_name=”my-model”,
    description=”my description”,
    labels={‘key’: ‘value’},

)

  • Parameters

    • display_name (str) – The display name of the Model. The name can be up to 128 characters long and can be consist of any UTF-8 characters.

    • description (str) – The description of the model.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The labels with user-defined metadata to organize your Models. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information and examples of labels.

  • Returns

    Updated model resource.

  • Return type

    model (aiplatform.Model)

  • Raises

    ValueError – If labels is not the correct format.

property update_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was last updated.

classmethod upload(serving_container_image_uri: Optional[str] = None, *, artifact_uri: Optional[str] = None, model_id: Optional[str] = None, parent_model: Optional[str] = None, is_default_version: bool = True, version_aliases: Optional[Sequence[str]] = None, version_description: Optional[str] = None, serving_container_predict_route: Optional[str] = None, serving_container_health_route: Optional[str] = None, description: Optional[str] = None, serving_container_command: Optional[Sequence[str]] = None, serving_container_args: Optional[Sequence[str]] = None, serving_container_environment_variables: Optional[Dict[str, str]] = None, serving_container_ports: Optional[Sequence[int]] = None, serving_container_grpc_ports: Optional[Sequence[int]] = None, local_model: Optional[LocalModel] = None, instance_schema_uri: Optional[str] = None, parameters_schema_uri: Optional[str] = None, prediction_schema_uri: Optional[str] = None, explanation_metadata: Optional[google.cloud.aiplatform_v1.types.explanation_metadata.ExplanationMetadata] = None, explanation_parameters: Optional[google.cloud.aiplatform_v1.types.explanation.ExplanationParameters] = None, display_name: Optional[str] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None, labels: Optional[Dict[str, str]] = None, encryption_spec_key_name: Optional[str] = None, staging_bucket: Optional[str] = None, sync=True, upload_request_timeout: Optional[float] = None, serving_container_deployment_timeout: Optional[int] = None, serving_container_shared_memory_size_mb: Optional[int] = None, serving_container_startup_probe_exec: Optional[Sequence[str]] = None, serving_container_startup_probe_period_seconds: Optional[int] = None, serving_container_startup_probe_timeout_seconds: Optional[int] = None, serving_container_health_probe_exec: Optional[Sequence[str]] = None, serving_container_health_probe_period_seconds: Optional[int] = None, serving_container_health_probe_timeout_seconds: Optional[int] = None)

Uploads a model and returns a Model representing the uploaded Model resource.

Example usage:

my_model = Model.upload(

    display_name=”my-model”,
    artifact_uri=”gs://my-model/saved-model”,
    serving_container_image_uri=”tensorflow/serving”

)

  • Parameters

    • serving_container_image_uri (str) – Optional. The URI of the Model serving container. This parameter is required if the parameter local_model is not specified.

    • artifact_uri (str) – Optional. The path to the directory containing the Model artifact and any of its supporting files. Leave blank for custom container prediction. Not present for AutoML Models.

    • model_id (str) – Optional. The ID to use for the uploaded Model, which will become the final component of the model resource name. This value may be up to 63 characters, and valid characters are [a-z0-9_-]. The first character cannot be a number or hyphen.

    • parent_model (str) – Optional. The resource name or model ID of an existing model that the newly-uploaded model will be a version of.

      Only set this field when uploading a new version of an existing model.

    • is_default_version (bool) – Optional. When set to True, the newly uploaded model version will automatically have alias “default” included. Subsequent uses of this model without a version specified will use this “default” version.

      When set to False, the “default” alias will not be moved. Actions targeting the newly-uploaded model version will need to specifically reference this version by ID or alias.

      New model uploads, i.e. version 1, will always be “default” aliased.

    • version_aliases (Sequence[str]) – Optional. User provided version aliases so that a model version can be referenced via alias instead of auto-generated version ID. A default version alias will be created for the first version of the model.

      The format is [a-z][a-zA-Z0-9-]{0,126}[a-z0-9]

    • version_description (str) – Optional. The description of the model version being uploaded.

    • serving_container_predict_route (str) – Optional. An HTTP path to send prediction requests to the container, and which must be supported by it. If not specified a default HTTP path will be used by Vertex AI.

    • serving_container_health_route (str) – Optional. An HTTP path to send health check requests to the container, and which must be supported by it. If not specified a standard HTTP path will be used by Vertex AI.

    • description (str) – The description of the model.

    • serving_container_command – Optional[Sequence[str]]=None, The command with which the container is run. Not executed within a shell. The Docker image’s ENTRYPOINT is used if this is not provided. Variable references $(VAR_NAME) are expanded using the container’s environment. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not.

    • serving_container_args – Optional[Sequence[str]]=None, The arguments to the command. The Docker image’s CMD is used if this is not provided. Variable references $(VAR_NAME) are expanded using the container’s environment. If a variable cannot be resolved, the reference in the input string will be unchanged. The $(VAR_NAME) syntax can be escaped with a double $$, ie: $$(VAR_NAME). Escaped references will never be expanded, regardless of whether the variable exists or not.

    • serving_container_environment_variables – Optional[Dict[str, str]]=None, The environment variables that are to be present in the container. Should be a dictionary where keys are environment variable names and values are environment variable values for those names.

    • serving_container_ports – Optional[Sequence[int]]=None, Declaration of ports that are exposed by the container. This field is primarily informational, it gives Vertex AI information about the network connections the container uses. Listing or not a port here has no impact on whether the port is actually exposed, any port listening on the default “0.0.0.0” address inside a container will be accessible from the network.

    • serving_container_grpc_ports – Optional[Sequence[int]]=None, Declaration of ports that are exposed by the container. Vertex AI sends gRPC prediction requests that it receives to the first port on this list. Vertex AI also sends liveness and health checks to this port. If you do not specify this field, gRPC requests to the container will be disabled. Vertex AI does not use ports other than the first one listed. This field corresponds to the ports field of the Kubernetes Containers v1 core API.

    • local_model (Optional[LocalModel]) – Optional. A LocalModel instance that includes a serving_container_spec. If provided, the serving_container_spec of the LocalModel instance will overwrite the values of all other serving container parameters.

    • instance_schema_uri (str) – Optional. Points to a YAML file stored on Google Cloud Storage describing the format of a single instance, which are used in PredictRequest.instances, ExplainRequest.instances and BatchPredictionJob.input_config. The schema is defined as an OpenAPI 3.0.2 Schema Object. AutoML Models always have this field populated by AI Platform. Note: The URI given on output will be immutable and probably different, including the URI scheme, than the one given on input. The output URI will point to a location where the user only has a read access.

    • parameters_schema_uri (str) – Optional. Points to a YAML file stored on Google Cloud Storage describing the parameters of prediction and explanation via PredictRequest.parameters, ExplainRequest.parameters and BatchPredictionJob.model_parameters. The schema is defined as an OpenAPI 3.0.2 Schema Object. AutoML Models always have this field populated by AI Platform, if no parameters are supported it is set to an empty string. Note: The URI given on output will be immutable and probably different, including the URI scheme, than the one given on input. The output URI will point to a location where the user only has a read access.

    • prediction_schema_uri (str) – Optional. Points to a YAML file stored on Google Cloud Storage describing the format of a single prediction produced by this Model, which are returned via PredictResponse.predictions, ExplainResponse.explanations, and BatchPredictionJob.output_config. The schema is defined as an OpenAPI 3.0.2 Schema Object. AutoML Models always have this field populated by AI Platform. Note: The URI given on output will be immutable and probably different, including the URI scheme, than the one given on input. The output URI will point to a location where the user only has a read access.

    • explanation_metadata (aiplatform.explain.ExplanationMetadata) – Optional. Metadata describing the Model’s input and output for explanation. explanation_metadata is optional while explanation_parameters must be specified when used. For more details, see Ref docs http://tinyurl.com/1igh60kt

    • explanation_parameters (aiplatform.explain.ExplanationParameters) – Optional. Parameters to configure explaining for Model’s predictions. For more details, see Ref docs http://tinyurl.com/1an4zake

    • display_name (str) – Optional. The display name of the Model. The name can be up to 128 characters long and can be consist of any UTF-8 characters.

    • project – Optional[str]=None, Project to upload this model to. Overrides project set in aiplatform.init.

    • location – Optional[str]=None, Location to upload this model to. Overrides location set in aiplatform.init.

    • credentials – Optional[auth_credentials.Credentials]=None, Custom credentials to use to upload this model. Overrides credentials set in aiplatform.init.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The labels with user-defined metadata to organize your Models. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information and examples of labels.

    • encryption_spec_key_name (Optional[str]) – Optional. The Cloud KMS resource identifier of the customer managed encryption key used to protect the model. Has the form: projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key. The key needs to be in the same region as where the compute resource is created.

      If set, this Model and all sub-resources of this Model will be secured by this key.

      Overrides encryption_spec_key_name set in aiplatform.init.

    • staging_bucket (str) – Optional. Bucket to stage local model artifacts. Overrides staging_bucket set in aiplatform.init.

    • upload_request_timeout (float) – Optional. The timeout for the upload request in seconds.

    • serving_container_deployment_timeout (int) – Optional. Deployment timeout in seconds.

    • serving_container_shared_memory_size_mb (int) – Optional. The amount of the VM memory to reserve as the shared memory for the model in megabytes.

    • serving_container_startup_probe_exec (Sequence[str]) – Optional. Exec specifies the action to take. Used by startup probe. An example of this argument would be [“cat”, “/tmp/healthy”]

    • serving_container_startup_probe_period_seconds (int) – Optional. How often (in seconds) to perform the startup probe. Default to 10 seconds. Minimum value is 1.

    • serving_container_startup_probe_timeout_seconds (int) – Optional. Number of seconds after which the startup probe times out. Defaults to 1 second. Minimum value is 1.

    • serving_container_health_probe_exec (Sequence[str]) – Optional. Exec specifies the action to take. Used by health probe. An example of this argument would be [“cat”, “/tmp/healthy”]

    • serving_container_health_probe_period_seconds (int) – Optional. How often (in seconds) to perform the health probe. Default to 10 seconds. Minimum value is 1.

    • serving_container_health_probe_timeout_seconds (int) – Optional. Number of seconds after which the health probe times out. Defaults to 1 second. Minimum value is 1.

  • Returns

    Instantiated representation of the uploaded model resource.

  • Return type

    model (aiplatform.Model)

  • Raises

    ValueError – If explanation_metadata is specified while explanation_parameters is not.

      Also if model directory does not contain a supported model file.
  If local_model is specified but serving_container_spec.image_uri
  in the local_model is None.
  If local_model is not specified and serving_container_image_uri
  is None.

classmethod upload_scikit_learn_model_file(model_file_path: str, sklearn_version: Optional[str] = None, display_name: Optional[str] = None, description: Optional[str] = None, model_id: Optional[str] = None, parent_model: Optional[str] = None, is_default_version: Optional[bool] = True, version_aliases: Optional[Sequence[str]] = None, version_description: Optional[str] = None, instance_schema_uri: Optional[str] = None, parameters_schema_uri: Optional[str] = None, prediction_schema_uri: Optional[str] = None, explanation_metadata: Optional[google.cloud.aiplatform_v1.types.explanation_metadata.ExplanationMetadata] = None, explanation_parameters: Optional[google.cloud.aiplatform_v1.types.explanation.ExplanationParameters] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None, labels: Optional[Dict[str, str]] = None, encryption_spec_key_name: Optional[str] = None, staging_bucket: Optional[str] = None, sync=True, upload_request_timeout: Optional[float] = None)

Uploads a model and returns a Model representing the uploaded Model resource.

Example usage:

my_model = Model.upload_scikit_learn_model_file(

    model_file_path=”iris.sklearn_model.joblib”

)

  • Parameters

    • model_file_path (str) – Required. Local file path of the model.

    • sklearn_version (str) – Optional. The version of the Scikit-learn serving container. Supported versions: [“0.20”, “0.22”, “0.23”, “0.24”, “1.0”]. If the version is not specified, the latest version is used.

    • display_name (str) – Optional. The display name of the Model. The name can be up to 128 characters long and can be consist of any UTF-8 characters.

    • description (str) – The description of the model.

    • model_id (str) – Optional. The ID to use for the uploaded Model, which will become the final component of the model resource name. This value may be up to 63 characters, and valid characters are [a-z0-9_-]. The first character cannot be a number or hyphen.

    • parent_model (str) – Optional. The resource name or model ID of an existing model that the newly-uploaded model will be a version of.

      Only set this field when uploading a new version of an existing model.

    • is_default_version (bool) – Optional. When set to True, the newly uploaded model version will automatically have alias “default” included. Subsequent uses of this model without a version specified will use this “default” version.

      When set to False, the “default” alias will not be moved. Actions targeting the newly-uploaded model version will need to specifically reference this version by ID or alias.

      New model uploads, i.e. version 1, will always be “default” aliased.

    • version_aliases (Sequence[str]) – Optional. User provided version aliases so that a model version can be referenced via alias instead of auto-generated version ID. A default version alias will be created for the first version of the model.

      The format is [a-z][a-zA-Z0-9-]{0,126}[a-z0-9]

    • version_description (str) – Optional. The description of the model version being uploaded.

    • instance_schema_uri (str) – Optional. Points to a YAML file stored on Google Cloud Storage describing the format of a single instance, which are used in PredictRequest.instances, ExplainRequest.instances and BatchPredictionJob.input_config. The schema is defined as an OpenAPI 3.0.2 Schema Object. AutoML Models always have this field populated by AI Platform. Note: The URI given on output will be immutable and probably different, including the URI scheme, than the one given on input. The output URI will point to a location where the user only has a read access.

    • parameters_schema_uri (str) – Optional. Points to a YAML file stored on Google Cloud Storage describing the parameters of prediction and explanation via PredictRequest.parameters, ExplainRequest.parameters and BatchPredictionJob.model_parameters. The schema is defined as an OpenAPI 3.0.2 Schema Object. AutoML Models always have this field populated by AI Platform, if no parameters are supported it is set to an empty string. Note: The URI given on output will be immutable and probably different, including the URI scheme, than the one given on input. The output URI will point to a location where the user only has a read access.

    • prediction_schema_uri (str) – Optional. Points to a YAML file stored on Google Cloud Storage describing the format of a single prediction produced by this Model, which are returned via PredictResponse.predictions, ExplainResponse.explanations, and BatchPredictionJob.output_config. The schema is defined as an OpenAPI 3.0.2 Schema Object. AutoML Models always have this field populated by AI Platform. Note: The URI given on output will be immutable and probably different, including the URI scheme, than the one given on input. The output URI will point to a location where the user only has a read access.

    • explanation_metadata (aiplatform.explain.ExplanationMetadata) – Optional. Metadata describing the Model’s input and output for explanation. explanation_metadata is optional while explanation_parameters must be specified when used. For more details, see Ref docs http://tinyurl.com/1igh60kt

    • explanation_parameters (aiplatform.explain.ExplanationParameters) – Optional. Parameters to configure explaining for Model’s predictions. For more details, see Ref docs http://tinyurl.com/1an4zake

    • project – Optional[str]=None, Project to upload this model to. Overrides project set in aiplatform.init.

    • location – Optional[str]=None, Location to upload this model to. Overrides location set in aiplatform.init.

    • credentials – Optional[auth_credentials.Credentials]=None, Custom credentials to use to upload this model. Overrides credentials set in aiplatform.init.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The labels with user-defined metadata to organize your Models. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information and examples of labels.

    • encryption_spec_key_name (Optional[str]) – Optional. The Cloud KMS resource identifier of the customer managed encryption key used to protect the model. Has the form: projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key. The key needs to be in the same region as where the compute resource is created.

      If set, this Model and all sub-resources of this Model will be secured by this key.

      Overrides encryption_spec_key_name set in aiplatform.init.

    • staging_bucket (str) – Optional. Bucket to stage local model artifacts. Overrides staging_bucket set in aiplatform.init.

    • sync (bool) – Whether to execute this method synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

    • upload_request_timeout (float) – Optional. The timeout for the upload request in seconds.

  • Returns

    Instantiated representation of the uploaded model resource.

  • Return type

    model (aiplatform.Model)

  • Raises

    ValueError – If explanation_metadata is specified while explanation_parameters is not. Also if model directory does not contain a supported model file.

classmethod upload_tensorflow_saved_model(saved_model_dir: str, tensorflow_version: Optional[str] = None, use_gpu: bool = False, display_name: Optional[str] = None, description: Optional[str] = None, model_id: Optional[str] = None, parent_model: Optional[str] = None, is_default_version: Optional[bool] = True, version_aliases: Optional[Sequence[str]] = None, version_description: Optional[str] = None, instance_schema_uri: Optional[str] = None, parameters_schema_uri: Optional[str] = None, prediction_schema_uri: Optional[str] = None, explanation_metadata: Optional[google.cloud.aiplatform_v1.types.explanation_metadata.ExplanationMetadata] = None, explanation_parameters: Optional[google.cloud.aiplatform_v1.types.explanation.ExplanationParameters] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None, labels: Optional[Dict[str, str]] = None, encryption_spec_key_name: Optional[str] = None, staging_bucket: Optional[str] = None, sync=True, upload_request_timeout: Optional[str] = None)

Uploads a model and returns a Model representing the uploaded Model resource.

Example usage:

my_model = Model.upload_scikit_learn_model_file(

    model_file_path=”iris.tensorflow_model.SavedModel”

)

  • Parameters

    • saved_model_dir (str) – Required. Local directory of the Tensorflow SavedModel.

    • tensorflow_version (str) – Optional. The version of the Tensorflow serving container. Supported versions: [“0.15”, “2.1”, “2.2”, “2.3”, “2.4”, “2.5”, “2.6”, “2.7”]. If the version is not specified, the latest version is used.

    • use_gpu (bool) – Whether to use GPU for model serving.

    • display_name (str) – Optional. The display name of the Model. The name can be up to 128 characters long and can be consist of any UTF-8 characters.

    • description (str) – The description of the model.

    • model_id (str) – Optional. The ID to use for the uploaded Model, which will become the final component of the model resource name. This value may be up to 63 characters, and valid characters are [a-z0-9_-]. The first character cannot be a number or hyphen.

    • parent_model (str) – Optional. The resource name or model ID of an existing model that the newly-uploaded model will be a version of.

      Only set this field when uploading a new version of an existing model.

    • is_default_version (bool) – Optional. When set to True, the newly uploaded model version will automatically have alias “default” included. Subsequent uses of this model without a version specified will use this “default” version.

      When set to False, the “default” alias will not be moved. Actions targeting the newly-uploaded model version will need to specifically reference this version by ID or alias.

      New model uploads, i.e. version 1, will always be “default” aliased.

    • version_aliases (Sequence[str]) – Optional. User provided version aliases so that a model version can be referenced via alias instead of auto-generated version ID. A default version alias will be created for the first version of the model.

      The format is [a-z][a-zA-Z0-9-]{0,126}[a-z0-9]

    • version_description (str) – Optional. The description of the model version being uploaded.

    • instance_schema_uri (str) – Optional. Points to a YAML file stored on Google Cloud Storage describing the format of a single instance, which are used in PredictRequest.instances, ExplainRequest.instances and BatchPredictionJob.input_config. The schema is defined as an OpenAPI 3.0.2 Schema Object. AutoML Models always have this field populated by AI Platform. Note: The URI given on output will be immutable and probably different, including the URI scheme, than the one given on input. The output URI will point to a location where the user only has a read access.

    • parameters_schema_uri (str) – Optional. Points to a YAML file stored on Google Cloud Storage describing the parameters of prediction and explanation via PredictRequest.parameters, ExplainRequest.parameters and BatchPredictionJob.model_parameters. The schema is defined as an OpenAPI 3.0.2 Schema Object. AutoML Models always have this field populated by AI Platform, if no parameters are supported it is set to an empty string. Note: The URI given on output will be immutable and probably different, including the URI scheme, than the one given on input. The output URI will point to a location where the user only has a read access.

    • prediction_schema_uri (str) – Optional. Points to a YAML file stored on Google Cloud Storage describing the format of a single prediction produced by this Model, which are returned via PredictResponse.predictions, ExplainResponse.explanations, and BatchPredictionJob.output_config. The schema is defined as an OpenAPI 3.0.2 Schema Object. AutoML Models always have this field populated by AI Platform. Note: The URI given on output will be immutable and probably different, including the URI scheme, than the one given on input. The output URI will point to a location where the user only has a read access.

    • explanation_metadata (aiplatform.explain.ExplanationMetadata) – Optional. Metadata describing the Model’s input and output for explanation. explanation_metadata is optional while explanation_parameters must be specified when used. For more details, see Ref docs http://tinyurl.com/1igh60kt

    • explanation_parameters (aiplatform.explain.ExplanationParameters) – Optional. Parameters to configure explaining for Model’s predictions. For more details, see Ref docs http://tinyurl.com/1an4zake

    • project – Optional[str]=None, Project to upload this model to. Overrides project set in aiplatform.init.

    • location – Optional[str]=None, Location to upload this model to. Overrides location set in aiplatform.init.

    • credentials – Optional[auth_credentials.Credentials]=None, Custom credentials to use to upload this model. Overrides credentials set in aiplatform.init.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The labels with user-defined metadata to organize your Models. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information and examples of labels.

    • encryption_spec_key_name (Optional[str]) – Optional. The Cloud KMS resource identifier of the customer managed encryption key used to protect the model. Has the form: projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key. The key needs to be in the same region as where the compute resource is created.

      If set, this Model and all sub-resources of this Model will be secured by this key.

      Overrides encryption_spec_key_name set in aiplatform.init.

    • staging_bucket (str) – Optional. Bucket to stage local model artifacts. Overrides staging_bucket set in aiplatform.init.

    • sync (bool) – Whether to execute this method synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

    • upload_request_timeout (float) – Optional. The timeout for the upload request in seconds.

  • Returns

    Instantiated representation of the uploaded model resource.

  • Return type

    model (aiplatform.Model)

  • Raises

    ValueError – If explanation_metadata is specified while explanation_parameters is not. Also if model directory does not contain a supported model file.

classmethod upload_xgboost_model_file(model_file_path: str, xgboost_version: Optional[str] = None, display_name: Optional[str] = None, description: Optional[str] = None, model_id: Optional[str] = None, parent_model: Optional[str] = None, is_default_version: Optional[bool] = True, version_aliases: Optional[Sequence[str]] = None, version_description: Optional[str] = None, instance_schema_uri: Optional[str] = None, parameters_schema_uri: Optional[str] = None, prediction_schema_uri: Optional[str] = None, explanation_metadata: Optional[google.cloud.aiplatform_v1.types.explanation_metadata.ExplanationMetadata] = None, explanation_parameters: Optional[google.cloud.aiplatform_v1.types.explanation.ExplanationParameters] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None, labels: Optional[Dict[str, str]] = None, encryption_spec_key_name: Optional[str] = None, staging_bucket: Optional[str] = None, sync=True, upload_request_timeout: Optional[float] = None)

Uploads a model and returns a Model representing the uploaded Model resource.

Example usage:

my_model = Model.upload_xgboost_model_file(

    model_file_path=”iris.xgboost_model.bst”

)

  • Parameters

    • model_file_path (str) – Required. Local file path of the model.

    • xgboost_version (str) – Optional. The version of the XGBoost serving container. Supported versions: [“0.82”, “0.90”, “1.1”, “1.2”, “1.3”, “1.4”]. If the version is not specified, the latest version is used.

    • display_name (str) – Optional. The display name of the Model. The name can be up to 128 characters long and can be consist of any UTF-8 characters.

    • description (str) – The description of the model.

    • model_id (str) – Optional. The ID to use for the uploaded Model, which will become the final component of the model resource name. This value may be up to 63 characters, and valid characters are [a-z0-9_-]. The first character cannot be a number or hyphen.

    • parent_model (str) – Optional. The resource name or model ID of an existing model that the newly-uploaded model will be a version of.

      Only set this field when uploading a new version of an existing model.

    • is_default_version (bool) – Optional. When set to True, the newly uploaded model version will automatically have alias “default” included. Subsequent uses of this model without a version specified will use this “default” version.

      When set to False, the “default” alias will not be moved. Actions targeting the newly-uploaded model version will need to specifically reference this version by ID or alias.

      New model uploads, i.e. version 1, will always be “default” aliased.

    • version_aliases (Sequence[str]) – Optional. User provided version aliases so that a model version can be referenced via alias instead of auto-generated version ID. A default version alias will be created for the first version of the model.

      The format is [a-z][a-zA-Z0-9-]{0,126}[a-z0-9]

    • version_description (str) – Optional. The description of the model version being uploaded.

    • instance_schema_uri (str) – Optional. Points to a YAML file stored on Google Cloud Storage describing the format of a single instance, which are used in PredictRequest.instances, ExplainRequest.instances and BatchPredictionJob.input_config. The schema is defined as an OpenAPI 3.0.2 Schema Object. AutoML Models always have this field populated by AI Platform. Note: The URI given on output will be immutable and probably different, including the URI scheme, than the one given on input. The output URI will point to a location where the user only has a read access.

    • parameters_schema_uri (str) – Optional. Points to a YAML file stored on Google Cloud Storage describing the parameters of prediction and explanation via PredictRequest.parameters, ExplainRequest.parameters and BatchPredictionJob.model_parameters. The schema is defined as an OpenAPI 3.0.2 Schema Object. AutoML Models always have this field populated by AI Platform, if no parameters are supported it is set to an empty string. Note: The URI given on output will be immutable and probably different, including the URI scheme, than the one given on input. The output URI will point to a location where the user only has a read access.

    • prediction_schema_uri (str) – Optional. Points to a YAML file stored on Google Cloud Storage describing the format of a single prediction produced by this Model, which are returned via PredictResponse.predictions, ExplainResponse.explanations, and BatchPredictionJob.output_config. The schema is defined as an OpenAPI 3.0.2 Schema Object. AutoML Models always have this field populated by AI Platform. Note: The URI given on output will be immutable and probably different, including the URI scheme, than the one given on input. The output URI will point to a location where the user only has a read access.

    • explanation_metadata (aiplatform.explain.ExplanationMetadata) – Optional. Metadata describing the Model’s input and output for explanation. explanation_metadata is optional while explanation_parameters must be specified when used. For more details, see Ref docs http://tinyurl.com/1igh60kt

    • explanation_parameters (aiplatform.explain.ExplanationParameters) – Optional. Parameters to configure explaining for Model’s predictions. For more details, see Ref docs http://tinyurl.com/1an4zake

    • project – Optional[str]=None, Project to upload this model to. Overrides project set in aiplatform.init.

    • location – Optional[str]=None, Location to upload this model to. Overrides location set in aiplatform.init.

    • credentials – Optional[auth_credentials.Credentials]=None, Custom credentials to use to upload this model. Overrides credentials set in aiplatform.init.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The labels with user-defined metadata to organize your Models. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information and examples of labels.

    • encryption_spec_key_name (Optional[str]) – Optional. The Cloud KMS resource identifier of the customer managed encryption key used to protect the model. Has the form: projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key. The key needs to be in the same region as where the compute resource is created.

      If set, this Model and all sub-resources of this Model will be secured by this key.

      Overrides encryption_spec_key_name set in aiplatform.init.

    • staging_bucket (str) – Optional. Bucket to stage local model artifacts. Overrides staging_bucket set in aiplatform.init.

    • upload_request_timeout (float) – Optional. The timeout for the upload request in seconds.

  • Returns

    Instantiated representation of the uploaded model resource.

  • Return type

    model (aiplatform.Model)

  • Raises

    ValueError – If model directory does not contain a supported model file.

property uri(: Optional[str )

Path to the directory containing the Model artifact and any of its supporting files. Not present for AutoML Models.

property version_aliases(: Sequence[str )

User provided version aliases so that a model version can be referenced via alias (i.e. projects/{project}/locations/{location}/models/{model_id}@{version_alias} instead of auto-generated version id (i.e. projects/{project}/locations/{location}/models/{model_id}@{version_id}). The format is [a-z][a-zA-Z0-9-]{0,126}[a-z0-9] to distinguish from version_id. A default version alias will be created for the first version of the model, and there must be exactly one default version alias for a model.

property version_create_time(: [google.protobuf.timestamp_pb2.Timestamp](https://googleapis.dev/python/protobuf/latest/google/protobuf/timestamp_pb2.html#google.protobuf.timestamp_pb2.Timestamp )

Timestamp when this version was created.

property version_description(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

The description of this version.

property version_id(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

The version ID of the model. A new version is committed when a new model version is uploaded or trained under an existing model id. It is an auto-incrementing decimal number in string representation.

property version_update_time(: [google.protobuf.timestamp_pb2.Timestamp](https://googleapis.dev/python/protobuf/latest/google/protobuf/timestamp_pb2.html#google.protobuf.timestamp_pb2.Timestamp )

Timestamp when this version was updated.

property versioned_resource_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

The fully-qualified resource name, including the version ID. For example, projects/{project}/locations/{location}/models/{model_id}@{version_id}

property versioning_registry(: google.cloud.aiplatform.models.ModelRegistr )

The registry of model versions associated with this Model instance.

wait()

Helper method that blocks until all futures are complete.

class google.cloud.aiplatform.ModelDeploymentMonitoringJob(model_deployment_monitoring_job_name: str, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Bases: google.cloud.aiplatform.jobs._Job

Vertex AI Model Deployment Monitoring Job.

This class should be used in conjunction with the Endpoint class in order to configure model monitoring for deployed models.

Initializer for ModelDeploymentMonitoringJob.

  • Parameters

    • model_deployment_monitoring_job_name (str) – Required. A fully-qualified ModelDeploymentMonitoringJob resource name or ID. Example: “projects/…/locations/…/modelDeploymentMonitoringJobs/456” or “456” when project and location are initialized or passed.

    • project – (str), Optional. project to retrieve ModelDeploymentMonitoringJob from. If not set, project set in aiplatform.init will be used.

    • location – (str), Optional. location to retrieve ModelDeploymentMonitoringJob from. If not set, location set in aiplatform.init will be used.

    • credentials – (auth_credentials.Credentials), Optional. Custom credentials to use. If not set, credentials set in aiplatform.init will be used.

classmethod cancel()

Cancels this Job.

Success of cancellation is not guaranteed. Use Job.state property to verify if cancellation was successful.

client_class()

alias of google.cloud.aiplatform.utils.JobClientWithOverride

classmethod create(endpoint: Union[str, google.cloud.aiplatform.models.Endpoint], objective_configs: Optional[Union[google.cloud.aiplatform.model_monitoring.objective.ObjectiveConfig, Dict[str, google.cloud.aiplatform.model_monitoring.objective.ObjectiveConfig]]] = None, logging_sampling_strategy: Optional[google.cloud.aiplatform.model_monitoring.sampling.RandomSampleConfig] = None, schedule_config: Optional[google.cloud.aiplatform.model_monitoring.schedule.ScheduleConfig] = None, display_name: Optional[str] = None, deployed_model_ids: Optional[List[str]] = None, alert_config: Optional[google.cloud.aiplatform.model_monitoring.alert.EmailAlertConfig] = None, predict_instance_schema_uri: Optional[str] = None, sample_predict_instance: Optional[str] = None, analysis_instance_schema_uri: Optional[str] = None, bigquery_tables_log_ttl: Optional[int] = None, stats_anomalies_base_directory: Optional[str] = None, enable_monitoring_pipeline_logs: Optional[bool] = None, labels: Optional[Dict[str, str]] = None, encryption_spec_key_name: Optional[str] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None, create_request_timeout: Optional[float] = None)

Creates and launches a model monitoring job.

  • Parameters

    • endpoint (Union[str, **"aiplatform.Endpoint"]) – Required. Endpoint resource name or an instance of aiplatform.Endpoint. Format: projects/{project}/locations/{location}/endpoints/{endpoint}

    • (Union[model_monitoring.ObjectiveConfig (objective_configs) – Dict[str, model_monitoring.ObjectiveConfig]]): Required. A single config if it applies to all models, or a dictionary of model_id: model_monitoring.objective.ObjectiveConfig if different model IDs have different configs.

:param

Required. A single config if it applies to all models, or a dictionary of
model_id: model_monitoring.objective.ObjectiveConfig if
different model IDs have different configs.

  • Parameters

    • logging_sampling_strategy (model_monitoring.sampling.RandomSampleConfig) – Optional. Sample Strategy for logging.

    • schedule_config (model_monitoring.schedule.ScheduleConfig) – Optional. Configures model monitoring job scheduling interval in hours. This defines how often the monitoring jobs are triggered.

    • display_name (str) – Optional. The user-defined name of the ModelDeploymentMonitoringJob. The name can be up to 128 characters long and can be consist of any UTF-8 characters. Display name of a ModelDeploymentMonitoringJob.

    • deployed_model_ids (List[str]) – Optional. Use this argument to specify which deployed models to apply the objective config to. If left unspecified, the same config will be applied to all deployed models.

    • alert_config (model_monitoring.alert.EmailAlertConfig) – Optional. Configures how alerts are sent to the user. Right now only email alert is supported.

    • predict_instance_schema_uri (str) – Optional. YAML schema file uri describing the format of a single instance, which are given to format the Endpoint’s prediction (and explanation). If not set, the schema will be generated from collected predict requests.

    • sample_predict_instance (str) – Optional. Sample Predict instance, same format as PredictionRequest.instances, this can be set as a replacement of predict_instance_schema_uri If not set, the schema will be generated from collected predict requests.

    • analysis_instance_schema_uri (str) – Optional. YAML schema file uri describing the format of a single instance that you want Tensorflow Data Validation (TFDV) to analyze. If this field is empty, all the feature data types are inferred from predict_instance_schema_uri, meaning that TFDV will use the data in the exact format as prediction request/response. If there are any data type differences between predict instance and TFDV instance, this field can be used to override the schema. For models trained with Vertex AI, this field must be set as all the fields in predict instance formatted as string.

    • bigquery_tables_log_ttl (int) – Optional. The TTL(time to live) of BigQuery tables in user projects which stores logs. A day is the basic unit of the TTL and we take the ceil of TTL/86400(a day). e.g. { second: 3600} indicates ttl = 1 day.

    • stats_anomalies_base_directory (str) – Optional. Stats anomalies base folder path.

    • enable_monitoring_pipeline_logs (bool) – Optional. If true, the scheduled monitoring pipeline logs are sent to Google Cloud Logging, including pipeline status and anomalies detected. Please note the logs incur cost, which are subject to Cloud Logging pricing.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The labels with user-defined metadata to organize the ModelDeploymentMonitoringJob. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information and examples of labels.

    • encryption_spec_key_name (str) – Optional. Customer-managed encryption key spec for a ModelDeploymentMonitoringJob. If set, this ModelDeploymentMonitoringJob and all sub-resources of this ModelDeploymentMonitoringJob will be secured by this key.

    • create_request_timeout (int) – Optional. Timeout in seconds for the model monitoring job creation request.

  • Returns

    An instance of ModelDeploymentMonitoringJob.

property create_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was created.

delete()

Deletes an MDM job.

property display_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Display name of this resource.

done()

Method indicating whether a job has completed.

  • Returns

    True if the job has completed.

property encryption_spec(: Optional[[google.cloud.aiplatform_v1.types.encryption_spec.EncryptionSpec](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.EncryptionSpec)_ )

Customer-managed encryption key options for this Vertex AI resource.

If this is set, then all resources created by this Vertex AI resource will be encrypted with the provided encryption key.

property end_time()

Time when the Job resource entered the JOB_STATE_SUCCEEDED, JOB_STATE_FAILED, or JOB_STATE_CANCELLED state.

property error(: Optional[google.rpc.status_pb2.Status )

Detailed error info for this Job resource. Only populated when the Job’s state is JOB_STATE_FAILED or JOB_STATE_CANCELLED.

property gca_resource(: [proto.message.Message](https://proto-plus-python.readthedocs.io/en/latest/reference/message.html#proto.message.Message )

The underlying resource proto representation.

property labels(: Dict[str, str )

User-defined labels containing metadata about this resource.

Read more about labels at https://goo.gl/xmQnxf

classmethod list(filter: Optional[str] = None, order_by: Optional[str] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

List all instances of this Job Resource.

Example Usage:

aiplatform.BatchPredictionJobs.list(

filter=’state=”JOB_STATE_SUCCEEDED” AND display_name=”my_job”’,

)

  • Parameters

    • filter (str) – Optional. An expression for filtering the results of the request. For field names both snake_case and camelCase are supported.

    • order_by (str) – Optional. A comma-separated list of fields to order by, sorted in ascending order. Use “desc” after a field name for descending. Supported fields: display_name, create_time, update_time

    • project (str) – Optional. Project to retrieve list from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to retrieve list from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to retrieve list. Overrides credentials set in aiplatform.init.

  • Returns

    List[VertexAiResourceNoun] - A list of Job resource objects.

property name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Name of this resource.

pause()

Pause a running MDM job.

property resource_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Full qualified resource name.

resume()

Resumes a paused MDM job.

property start_time(: Optional[datetime.datetime )

Time when the Job resource entered the JOB_STATE_RUNNING for the first time.

property state(: [google.cloud.aiplatform_v1.types.job_state.JobState](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.JobState_ )

Fetch Job again and return the current JobState.

  • Returns

    Enum that describes the state of a Vertex AI job.

  • Return type

    state (job_state.JobState)

to_dict()

Returns the resource proto as a dictionary.

update(*, display_name: Optional[str] = None, schedule_config: Optional[google.cloud.aiplatform.model_monitoring.schedule.ScheduleConfig] = None, alert_config: Optional[google.cloud.aiplatform.model_monitoring.alert.EmailAlertConfig] = None, logging_sampling_strategy: Optional[google.cloud.aiplatform.model_monitoring.sampling.RandomSampleConfig] = None, labels: Optional[Dict[str, str]] = None, bigquery_tables_log_ttl: Optional[int] = None, enable_monitoring_pipeline_logs: Optional[bool] = None, objective_configs: Optional[Union[google.cloud.aiplatform.model_monitoring.objective.ObjectiveConfig, Dict[str, google.cloud.aiplatform.model_monitoring.objective.ObjectiveConfig]]] = None, deployed_model_ids: Optional[List[str]] = None, update_request_timeout: Optional[float] = None)

Updates an existing ModelDeploymentMonitoringJob.

  • Parameters

    • display_name (str) – Optional. The user-defined name of the ModelDeploymentMonitoringJob. The name can be up to 128 characters long and can be consist of any UTF-8 characters. Display name of a ModelDeploymentMonitoringJob.

    • schedule_config (model_monitoring.schedule.ScheduleConfig) – Required. Configures model monitoring job scheduling interval in hours. This defines how often the monitoring jobs are triggered.

    • alert_config (model_monitoring.alert.EmailAlertConfig) – Optional. Configures how alerts are sent to the user. Right now only email alert is supported.

    • logging_sampling_strategy (model_monitoring.sampling.RandomSampleConfig) – Required. Sample Strategy for logging.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The labels with user-defined metadata to organize the ModelDeploymentMonitoringJob. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information and examples of labels.

    • bigquery_tables_log_ttl (int) – Optional. The number of days for which the logs are stored. The TTL(time to live) of BigQuery tables in user projects which stores logs. A day is the basic unit of the TTL and we take the ceil of TTL/86400(a day). e.g. { second: 3600} indicates ttl = 1 day.

    • enable_monitoring_pipeline_logs (bool) – Optional. If true, the scheduled monitoring pipeline logs are sent to Google Cloud Logging, including pipeline status and anomalies detected. Please note the logs incur cost, which are subject to Cloud Logging pricing.

    • (Union[ (objective_configs) – Required. model_monitoring.objective.ObjectiveConfig, Dict[str, model_monitoring.objective.ObjectiveConfig]): A single config if it applies to all models, or a dictionary of model_id: model_monitoring.objective.ObjectiveConfig if different model IDs have different configs.

    • deployed_model_ids (List[str]) – Optional. Use this argument to specify which deployed models to apply the updated objective config to. If left unspecified, the same config will be applied to all deployed models.

    • upate_request_timeout (float) – Optional. Timeout in seconds for the model monitoring job update request.

property update_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was last updated.

wait()

Helper method that blocks until all futures are complete.

wait_for_completion()

Waits for job to complete.

class google.cloud.aiplatform.ModelEvaluation(evaluation_name: str, model_id: Optional[str] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Bases: google.cloud.aiplatform.base.VertexAiResourceNounWithFutureManager

Retrieves the ModelEvaluation resource and instantiates its representation.

  • Parameters

    • evaluation_name (str) – Required. A fully-qualified model evaluation resource name or evaluation ID. Example: “projects/123/locations/us-central1/models/456/evaluations/789” or “789”. If passing only the evaluation ID, model_id must be provided.

    • model_id (str) – Optional. The ID of the model to retrieve this evaluation from. If passing only the evaluation ID as evaluation_name, model_id must be provided.

    • project (str) – Optional project to retrieve model evaluation from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional location to retrieve model evaluation from. If not set, location set in aiplatform.init will be used.

    • credentials – Optional[auth_credentials.Credentials]=None, Custom credentials to use to retrieve this model evaluation. If not set, credentials set in aiplatform.init will be used.

client_class()

alias of google.cloud.aiplatform.utils.ModelClientWithOverride

property create_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was created.

delete()

Deletes this Vertex AI resource. WARNING: This deletion is permanent.

  • Parameters

    sync (bool) – Whether to execute this deletion synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

property display_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Display name of this resource.

property encryption_spec(: Optional[[google.cloud.aiplatform_v1.types.encryption_spec.EncryptionSpec](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.EncryptionSpec)_ )

Customer-managed encryption key options for this Vertex AI resource.

If this is set, then all resources created by this Vertex AI resource will be encrypted with the provided encryption key.

property gca_resource(: [proto.message.Message](https://proto-plus-python.readthedocs.io/en/latest/reference/message.html#proto.message.Message )

The underlying resource proto representation.

property labels(: Dict[str, str )

User-defined labels containing metadata about this resource.

Read more about labels at https://goo.gl/xmQnxf

classmethod list(model: str, filter: Optional[str] = None, order_by: Optional[str] = None, enable_simple_view: bool = False, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

List all ModelEvaluation resources on the provided model.

Example Usage:

aiplatform.ModelEvaluation.list(

model=”projects/123/locations/us-central1/models/456”,

)

aiplatform.Model.list(

model=”projects/123/locations/us-central1/models/456”,
order_by=”create_time desc, display_name”

)

  • Parameters

    • model (str) – Required. The resource name of the model to list evaluations for. For example: “projects/123/locations/us-central1/models/456”.

    • filter (str) – Optional. An expression for filtering the results of the request. For field names both snake_case and camelCase are supported.

    • order_by (str) – Optional. A comma-separated list of fields to order by, sorted in ascending order. Use “desc” after a field name for descending. Supported fields: display_name, create_time, update_time

    • project (str) – Optional. Project to retrieve list from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to retrieve list from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to retrieve list. Overrides credentials set in aiplatform.init.

    • parent (str) – Optional. The parent resource name if any to retrieve list from.

  • Returns

    List[VertexAiResourceNoun] - A list of SDK resource objects

property metrics(: [google.protobuf.struct_pb2.Value](https://googleapis.dev/python/protobuf/latest/google/protobuf/struct_pb2.html#google.protobuf.struct_pb2.Value )

Gets the evaluation metrics from the Model Evaluation.

  • Returns

    A struct_pb2.Value with model metrics created from the Model Evaluation

  • Raises

    ValueError – If the Model Evaluation doesn’t have metrics.

property name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Name of this resource.

property resource_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Full qualified resource name.

to_dict()

Returns the resource proto as a dictionary.

property update_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was last updated.

wait()

Helper method that blocks until all futures are complete.

class google.cloud.aiplatform.PipelineJob(display_name: str, template_path: str, job_id: Optional[str] = None, pipeline_root: Optional[str] = None, parameter_values: Optional[Dict[str, Any]] = None, input_artifacts: Optional[Dict[str, str]] = None, enable_caching: Optional[bool] = None, encryption_spec_key_name: Optional[str] = None, labels: Optional[Dict[str, str]] = None, credentials: Optional[google.auth.credentials.Credentials] = None, project: Optional[str] = None, location: Optional[str] = None, failure_policy: Optional[str] = None)

Bases: google.cloud.aiplatform.base.VertexAiStatefulResource, google.cloud.aiplatform.metadata.experiment_resources._ExperimentLoggable

Retrieves a PipelineJob resource and instantiates its representation.

  • Parameters

    • display_name (str) – Required. The user-defined name of this Pipeline.

    • template_path (str) – Required. The path of PipelineJob or PipelineSpec JSON or YAML file. It can be a local path, a Google Cloud Storage URI (e.g. “gs://project.name”), an Artifact Registry URI (e.g. “https://us-central1-kfp.pkg.dev/proj/repo/pack/latest”), or an HTTPS URI.

    • job_id (str) – Optional. The unique ID of the job run. If not specified, pipeline name + timestamp will be used.

    • pipeline_root (str) – Optional. The root of the pipeline outputs. If not set, the staging bucket set in aiplatform.init will be used. If that’s not set a pipeline-specific artifacts bucket will be used.

    • parameter_values (Dict[str, **Any]) – Optional. The mapping from runtime parameter names to its values that control the pipeline run.

    • input_artifacts (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The mapping from the runtime parameter name for this artifact to its resource id. For example: “vertex_model”:”456”. Note: full resource name (“projects/123/locations/us-central1/metadataStores/default/artifacts/456”) cannot be used.

    • enable_caching (bool) – Optional. Whether to turn on caching for the run.

      If this is not set, defaults to the compile time settings, which are True for all tasks by default, while users may specify different caching options for individual tasks.

      If this is set, the setting applies to all tasks in the pipeline.

      Overrides the compile time settings.

    • encryption_spec_key_name (str) – Optional. The Cloud KMS resource identifier of the customer managed encryption key used to protect the job. Has the form: projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key. The key needs to be in the same region as where the compute resource is created.

      If this is set, then all resources created by the PipelineJob will be encrypted with the provided encryption key.

      Overrides encryption_spec_key_name set in aiplatform.init.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The user defined metadata to organize PipelineJob.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to create this PipelineJob. Overrides credentials set in aiplatform.init.

    • project (str) – Optional. The project that you want to run this PipelineJob in. If not set, the project set in aiplatform.init will be used.

    • location (str) – Optional. Location to create PipelineJob. If not set, location set in aiplatform.init will be used.

    • failure_policy (str) – Optional. The failure policy - “slow” or “fast”. Currently, the default of a pipeline is that the pipeline will continue to run until no more tasks can be executed, also known as PIPELINE_FAILURE_POLICY_FAIL_SLOW (corresponds to “slow”). However, if a pipeline is set to PIPELINE_FAILURE_POLICY_FAIL_FAST (corresponds to “fast”), it will stop scheduling any new tasks when a task has failed. Any scheduled tasks will continue to completion.

  • Raises

    ValueError – If job_id or labels have incorrect format.

classmethod _init_subclass_(*, experiment_loggable_schemas: Tuple[google.cloud.aiplatform.metadata.experiment_resources._ExperimentLoggableSchema], **kwargs)

Register the metadata_schema for the subclass so Experiment can use it to retrieve the associated types.

usage:

class PipelineJob(…, experiment_loggable_schemas=

(_ExperimentLoggableSchema(title=’system.PipelineRun’), )

  • Parameters

    experiment_loggable_schemas – Tuple of the schema_title and type pairs that represent this resource. Note that a single item in the tuple will be most common. Currently only experiment run has multiple representation for backwards compatibility. Almost all schemas should be Contexts and Execution is currently only supported for backwards compatibility of experiment runs.

batch_cancel(project: str, location: str, names: List[str])

Example Usage:

> pipeline_job = aiplatform.PipelineJob(

>     display_name=’job_display_name’,
>     template_path=’your_pipeline.yaml’,

)

    pipeline_job.batch_cancel(

        project=’your_project_id’,
        location=’your_location’,
        names=[‘pipeline_job_name’,

    ‘pipeline_job_name2’]
    )

  • Parameters

    • project – Required. The project id of the PipelineJobs to batch delete.

    • location – Required. The location of the PipelineJobs to batch delete.

    • names – Required. The names of the PipelineJobs to cancel. A maximum of 32 PipelineJobs can be cancelled in a batch.

  • Returns

    An object representing a long-running operation.

  • Return type

    operation (Operation)

batch_delete(project: str, location: str, names: List[str])

Example Usage:

> pipeline_job = aiplatform.PipelineJob(

>     display_name=’job_display_name’,
>     template_path=’your_pipeline.yaml’,

)

    pipeline_job.batch_delete(

        project=’your_project_id’,
        location=’your_location’,
        names=[‘pipeline_job_name’,

    ‘pipeline_job_name2’]
    )

  • Parameters

    • project – Required. The project id of the PipelineJobs to batch delete.

    • location – Required. The location of the PipelineJobs to batch delete.

    • names – Required. The names of the PipelineJobs to delete. A maximum of 32 PipelineJobs can be deleted in a batch.

  • Returns

    BatchDeletePipelineJobsResponse contains PipelineJobs deleted.

cancel()

Starts asynchronous cancellation on the PipelineJob. The server makes a best effort to cancel the job, but success is not guaranteed. On successful cancellation, the PipelineJob is not deleted; instead it becomes a job with state set to CANCELLED.

client_class()

alias of google.cloud.aiplatform.utils.PipelineJobClientWithOverride

clone(display_name: Optional[str] = None, job_id: Optional[str] = None, pipeline_root: Optional[str] = None, parameter_values: Optional[Dict[str, Any]] = None, input_artifacts: Optional[Dict[str, str]] = None, enable_caching: Optional[bool] = None, encryption_spec_key_name: Optional[str] = None, labels: Optional[Dict[str, str]] = None, credentials: Optional[google.auth.credentials.Credentials] = None, project: Optional[str] = None, location: Optional[str] = None)

Returns a new PipelineJob object with the same settings as the original one.

  • Parameters

    • display_name (str) – Optional. The user-defined name of this cloned Pipeline. If not specified, original pipeline display name will be used.

    • job_id (str) – Optional. The unique ID of the job run. If not specified, “cloned” + pipeline name + timestamp will be used.

    • pipeline_root (str) – Optional. The root of the pipeline outputs. Default to be the same staging bucket as original pipeline.

    • parameter_values (Dict[str, **Any]) – Optional. The mapping from runtime parameter names to its values that control the pipeline run. Defaults to be the same values as original PipelineJob.

    • input_artifacts (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The mapping from the runtime parameter name for this artifact to its resource id. Defaults to be the same values as original PipelineJob. For example: “vertex_model”:”456”. Note: full resource name (“projects/123/locations/us-central1/metadataStores/default/artifacts/456”) cannot be used.

    • enable_caching (bool) – Optional. Whether to turn on caching for the run. If this is not set, defaults to be the same as original pipeline. If this is set, the setting applies to all tasks in the pipeline.

    • encryption_spec_key_name (str) – Optional. The Cloud KMS resource identifier of the customer managed encryption key used to protect the job. Has the form: projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key. The key needs to be in the same region as where the compute resource is created. If this is set, then all resources created by the PipelineJob will be encrypted with the provided encryption key. If not specified, encryption_spec of original PipelineJob will be used.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The user defined metadata to organize PipelineJob.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to create this PipelineJob. Overrides credentials set in aiplatform.init.

    • project (str) – Optional. The project that you want to run this PipelineJob in. If not set, the project set in original PipelineJob will be used.

    • location (str) – Optional. Location to create PipelineJob. If not set, location set in original PipelineJob will be used.

  • Returns

    A Vertex AI PipelineJob.

  • Raises

    ValueError – If job_id or labels have incorrect format.

create_schedule(cron: str, display_name: str, start_time: Optional[str] = None, end_time: Optional[str] = None, allow_queueing: bool = False, max_run_count: Optional[int] = None, max_concurrent_run_count: int = 1, service_account: Optional[str] = None, network: Optional[str] = None, create_request_timeout: Optional[float] = None)

Creates a PipelineJobSchedule directly from a PipelineJob.

Example Usage:

pipeline_job = aiplatform.PipelineJob(

display_name=’job_display_name’,
template_path=’your_pipeline.yaml’,

) pipeline_job.run() pipeline_job_schedule = pipeline_job.create_schedule(

cron=’* * * * 

*

’, display_name=’schedule_display_name’,

)

  • Parameters

    • cron (str) – Required. Time specification (cron schedule expression) to launch scheduled runs. To explicitly set a timezone to the cron tab, apply a prefix: “CRON_TZ=${IANA_TIME_ZONE}” or “TZ=${IANA_TIME_ZONE}”. The ${IANA_TIME_ZONE} may only be a valid string from IANA time zone database. For example, “CRON_TZ=America/New_York 1 * * * 

      *

      ”, or “TZ=America/New_York 1 * * * 

      *

      ”.

    • display_name (str) – Required. The user-defined name of this PipelineJobSchedule.

    • start_time (str) – Optional. Timestamp after which the first run can be scheduled. If unspecified, it defaults to the schedule creation timestamp.

    • end_time (str) – Optional. Timestamp after which no more runs will be scheduled. If unspecified, then runs will be scheduled indefinitely.

    • allow_queueing (bool) – Optional. Whether new scheduled runs can be queued when max_concurrent_runs limit is reached.

    • max_run_count (int) – Optional. Maximum run count of the schedule. If specified, The schedule will be completed when either started_run_count >= max_run_count or when end_time is reached. Must be positive and <= 2^63-1.

    • max_concurrent_run_count (int) – Optional. Maximum number of runs that can be started concurrently for this PipelineJobSchedule.

    • service_account (str) – Optional. Specifies the service account for workload run-as account. Users submitting jobs must have act-as permission on this run-as account.

    • network (str) – Optional. The full name of the Compute Engine network to which the job should be peered. For example, projects/12345/global/networks/myVPC. Private services access must already be configured for the network. If left unspecified, the network set in aiplatform.init will be used. Otherwise, the job is not peered with any network.

    • create_request_timeout (float) – Optional. The timeout for the create request in seconds.

  • Returns

    A Vertex AI PipelineJobSchedule.

property create_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was created.

delete(sync: bool = True)

Deletes this Vertex AI resource. WARNING: This deletion is permanent.

  • Parameters

    sync (bool) – Whether to execute this deletion synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

property display_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Display name of this resource.

done()

Helper method that return True is PipelineJob is done. False otherwise.

property encryption_spec(: Optional[[google.cloud.aiplatform_v1.types.encryption_spec.EncryptionSpec](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.EncryptionSpec)_ )

Customer-managed encryption key options for this Vertex AI resource.

If this is set, then all resources created by this Vertex AI resource will be encrypted with the provided encryption key.

static from_pipeline_func(pipeline_func: Callable, parameter_values: Optional[Dict[str, Any]] = None, input_artifacts: Optional[Dict[str, str]] = None, output_artifacts_gcs_dir: Optional[str] = None, enable_caching: Optional[bool] = None, context_name: Optional[str] = 'pipeline', display_name: Optional[str] = None, labels: Optional[Dict[str, str]] = None, job_id: Optional[str] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None, encryption_spec_key_name: Optional[str] = None)

Creates PipelineJob by compiling a pipeline function.

  • Parameters

    • pipeline_func (Callable) – Required. A pipeline function to compile. A pipeline function creates instances of components and connects component inputs to outputs.

    • parameter_values (Dict[str, **Any]) – Optional. The mapping from runtime parameter names to its values that control the pipeline run.

    • input_artifacts (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The mapping from the runtime parameter name for this artifact to its resource id. For example: “vertex_model”:”456”. Note: full resource name (“projects/123/locations/us-central1/metadataStores/default/artifacts/456”) cannot be used.

    • output_artifacts_gcs_dir (str) – Optional. The GCS location of the pipeline outputs. A GCS bucket for artifacts will be created if not specified.

    • enable_caching (bool) – Optional. Whether to turn on caching for the run.

      If this is not set, defaults to the compile time settings, which are True for all tasks by default, while users may specify different caching options for individual tasks.

      If this is set, the setting applies to all tasks in the pipeline.

      Overrides the compile time settings.

    • context_name (str) – Optional. The name of metadata context. Used for cached execution reuse.

    • display_name (str) – Optional. The user-defined name of this Pipeline.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The user defined metadata to organize PipelineJob.

    • job_id (str) – Optional. The unique ID of the job run. If not specified, pipeline name + timestamp will be used.

    • project (str) – Optional. The project that you want to run this PipelineJob in. If not set, the project set in aiplatform.init will be used.

    • location (str) – Optional. Location to create PipelineJob. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to create this PipelineJob. Overrides credentials set in aiplatform.init.

    • encryption_spec_key_name (str) – Optional. The Cloud KMS resource identifier of the customer managed encryption key used to protect the job. Has the form: projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key. The key needs to be in the same region as where the compute resource is created.

      If this is set, then all resources created by the PipelineJob will be encrypted with the provided encryption key.

      Overrides encryption_spec_key_name set in aiplatform.init.

  • Returns

    A Vertex AI PipelineJob.

  • Raises

    ValueError – If job_id or labels have incorrect format.

property gca_resource(: [proto.message.Message](https://proto-plus-python.readthedocs.io/en/latest/reference/message.html#proto.message.Message )

The underlying resource proto representation.

classmethod get(resource_name: str, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Get a Vertex AI Pipeline Job for the given resource_name.

  • Parameters

    • resource_name (str) – Required. A fully-qualified resource name or ID.

    • project (str) – Optional. Project to retrieve dataset from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to retrieve dataset from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to upload this model. Overrides credentials set in aiplatform.init.

  • Returns

    A Vertex AI PipelineJob.

get_associated_experiment()

Gets the aiplatform.Experiment associated with this PipelineJob, or None if this PipelineJob is not associated with an experiment.

  • Returns

    An aiplatform.Experiment resource or None if this PipelineJob is not associated with an experiment..

property has_failed(: [bool](https://python.readthedocs.io/en/latest/library/functions.html#bool )

Returns True if pipeline has failed.

False otherwise.

property labels(: Dict[str, str )

User-defined labels containing metadata about this resource.

Read more about labels at https://goo.gl/xmQnxf

classmethod list(filter: Optional[str] = None, order_by: Optional[str] = None, enable_simple_view: bool = False, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

List all instances of this PipelineJob resource.

Example Usage:

aiplatform.PipelineJob.list(

filter=’display_name=”experiment_a27”’,
order_by=’create_time desc’

)

  • Parameters

    • filter (str) – Optional. An expression for filtering the results of the request. For field names both snake_case and camelCase are supported.

    • order_by (str) – Optional. A comma-separated list of fields to order by, sorted in ascending order. Use “desc” after a field name for descending. Supported fields: display_name, create_time, update_time

    • enable_simple_view (bool) – Optional. Whether to pass the read_mask parameter to the list call. Defaults to False if not provided. This will improve the performance of calling list(). However, the returned PipelineJob list will not include all fields for each PipelineJob. Setting this to True will exclude the following fields in your response: runtime_config, service_account, network, and some subfields of pipeline_spec and job_detail. The following fields will be included in each PipelineJob resource in your response: state, display_name, pipeline_spec.pipeline_info, create_time, start_time, end_time, update_time, labels, template_uri, template_metadata.version, job_detail.pipeline_run_context, job_detail.pipeline_context.

    • project (str) – Optional. Project to retrieve list from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to retrieve list from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to retrieve list. Overrides credentials set in aiplatform.init.

  • Returns

    List[PipelineJob] - A list of PipelineJob resource objects

property name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Name of this resource.

property resource_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Full qualified resource name.

run(service_account: Optional[str] = None, network: Optional[str] = None, reserved_ip_ranges: Optional[List[str]] = None, sync: Optional[bool] = True, create_request_timeout: Optional[float] = None)

Run this configured PipelineJob and monitor the job until completion.

  • Parameters

    • service_account (str) – Optional. Specifies the service account for workload run-as account. Users submitting jobs must have act-as permission on this run-as account.

    • network (str) – Optional. The full name of the Compute Engine network to which the job should be peered. For example, projects/12345/global/networks/myVPC. Private services access must already be configured for the network. If left unspecified, the network set in aiplatform.init will be used. Otherwise, the job is not peered with any network.

    • reserved_ip_ranges (List[str]) – Optional. A list of names for the reserved IP ranges under the VPC network that can be used for this PipelineJob’s workload. For example: [‘vertex-ai-ip-range’].

    • sync (bool) – Optional. Whether to execute this method synchronously. If False, this method will unblock and it will be executed in a concurrent Future.

    • create_request_timeout (float) – Optional. The timeout for the create request in seconds.

property state(: Optional[[google.cloud.aiplatform_v1.types.pipeline_state.PipelineState](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.PipelineState)_ )

Current pipeline state.

submit(service_account: Optional[str] = None, network: Optional[str] = None, reserved_ip_ranges: Optional[List[str]] = None, create_request_timeout: Optional[float] = None, *, experiment: Optional[Union[google.cloud.aiplatform.metadata.experiment_resources.Experiment, str]] = None)

Run this configured PipelineJob.

  • Parameters

    • service_account (str) – Optional. Specifies the service account for workload run-as account. Users submitting jobs must have act-as permission on this run-as account.

    • network (str) – Optional. The full name of the Compute Engine network to which the job should be peered. For example, projects/12345/global/networks/myVPC.

      Private services access must already be configured for the network. If left unspecified, the network set in aiplatform.init will be used. Otherwise, the job is not peered with any network.

    • reserved_ip_ranges (List[str]) – Optional. A list of names for the reserved IP ranges under the VPC network that can be used for this PipelineJob’s workload. For example: [‘vertex-ai-ip-range’].

      If left unspecified, the job will be deployed to any IP ranges under the provided VPC network.

    • create_request_timeout (float) – Optional. The timeout for the create request in seconds.

    • experiment (Union[str, **experiments_resource.Experiment]) – Optional. The Vertex AI experiment name or instance to associate to this PipelineJob.

      Metrics produced by the PipelineJob as system.Metric Artifacts will be associated as metrics to the current Experiment Run.

      Pipeline parameters will be associated as parameters to the current Experiment Run.

to_dict()

Returns the resource proto as a dictionary.

property update_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was last updated.

wait()

Wait for this PipelineJob to complete.

wait_for_resource_creation()

Waits until resource has been created.

class google.cloud.aiplatform.PipelineJobSchedule(pipeline_job: google.cloud.aiplatform.pipeline_jobs.PipelineJob, display_name: str, credentials: Optional[google.auth.credentials.Credentials] = None, project: Optional[str] = None, location: Optional[str] = None)

Bases: google.cloud.aiplatform.schedules._Schedule

Retrieves a PipelineJobSchedule resource and instantiates its representation.

  • Parameters

    • pipeline_job (PipelineJob) – Required. PipelineJob used to init the schedule.

    • display_name (str) – Required. The user-defined name of this PipelineJobSchedule.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to create this PipelineJobSchedule. Overrides credentials set in aiplatform.init.

    • project (str) – Optional. The project that you want to run this PipelineJobSchedule in. If not set, the project used for the PipelineJob will be used.

    • location (str) – Optional. Location to create PipelineJobSchedule. If not set, location used for the PipelineJob will be used.

property allow_queueing(: [bool](https://python.readthedocs.io/en/latest/library/functions.html#bool )

Whether current Schedule allows queueing.

  • Returns

    Schedule allow_queueing.

client_class()

alias of google.cloud.aiplatform.utils.ScheduleClientWithOverride

create(cron: str, start_time: Optional[str] = None, end_time: Optional[str] = None, allow_queueing: bool = False, max_run_count: Optional[int] = None, max_concurrent_run_count: int = 1, service_account: Optional[str] = None, network: Optional[str] = None, create_request_timeout: Optional[float] = None)

Create a PipelineJobSchedule.

  • Parameters

    • cron (str) – Required. Time specification (cron schedule expression) to launch scheduled runs. To explicitly set a timezone to the cron tab, apply a prefix: “CRON_TZ=${IANA_TIME_ZONE}” or “TZ=${IANA_TIME_ZONE}”. The ${IANA_TIME_ZONE} may only be a valid string from IANA time zone database. For example, “CRON_TZ=America/New_York 1 * * * 

      *

      ”, or “TZ=America/New_York 1 * * * 

      *

      ”.

    • start_time (str) – Optional. Timestamp after which the first run can be scheduled. If unspecified, it defaults to the schedule creation timestamp.

    • end_time (str) – Optional. Timestamp after which no more runs will be scheduled. If unspecified, then runs will be scheduled indefinitely.

    • allow_queueing (bool) – Optional. Whether new scheduled runs can be queued when max_concurrent_runs limit is reached.

    • max_run_count (int) – Optional. Maximum run count of the schedule. If specified, The schedule will be completed when either started_run_count >= max_run_count or when end_time is reached. Must be positive and <= 2^63-1.

    • max_concurrent_run_count (int) – Optional. Maximum number of runs that can be started concurrently for this PipelineJobSchedule.

    • service_account (str) – Optional. Specifies the service account for workload run-as account. Users submitting jobs must have act-as permission on this run-as account.

    • network (str) – Optional. The full name of the Compute Engine network to which the job should be peered. For example, projects/12345/global/networks/myVPC. Private services access must already be configured for the network. If left unspecified, the network set in aiplatform.init will be used. Otherwise, the job is not peered with any network.

    • create_request_timeout (float) – Optional. The timeout for the create request in seconds.

property create_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was created.

property cron(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Current Schedule cron.

  • Returns

    Schedule cron.

delete(sync: bool = True)

Deletes this Vertex AI resource. WARNING: This deletion is permanent.

  • Parameters

    sync (bool) – Whether to execute this deletion synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

property display_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Display name of this resource.

done()

Helper method that return True is Schedule is done. False otherwise.

property encryption_spec(: Optional[[google.cloud.aiplatform_v1.types.encryption_spec.EncryptionSpec](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.EncryptionSpec)_ )

Customer-managed encryption key options for this Vertex AI resource.

If this is set, then all resources created by this Vertex AI resource will be encrypted with the provided encryption key.

property gca_resource(: [proto.message.Message](https://proto-plus-python.readthedocs.io/en/latest/reference/message.html#proto.message.Message )

The underlying resource proto representation.

classmethod get(schedule_id: str, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Get a Vertex AI Schedule for the given resource_name.

  • Parameters

    • schedule_id (str) – Required. Schedule ID used to identify or locate the schedule.

    • project (str) – Optional. Project to retrieve dataset from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to retrieve dataset from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to upload this model. Overrides credentials set in aiplatform.init.

  • Returns

    A Vertex AI Schedule.

property labels(: Dict[str, str )

User-defined labels containing metadata about this resource.

Read more about labels at https://goo.gl/xmQnxf

classmethod list(filter: Optional[str] = None, order_by: Optional[str] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

List all instances of this PipelineJobSchedule resource.

Example Usage:

aiplatform.PipelineJobSchedule.list(

filter=’display_name=”experiment_a27”’,
order_by=’create_time desc’

)

  • Parameters

    • filter (str) – Optional. An expression for filtering the results of the request. For field names both snake_case and camelCase are supported.

    • order_by (str) – Optional. A comma-separated list of fields to order by, sorted in ascending order. Use “desc” after a field name for descending. Supported fields: display_name, create_time, update_time

    • project (str) – Optional. Project to retrieve list from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to retrieve list from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to retrieve list. Overrides credentials set in aiplatform.init.

  • Returns

    List[PipelineJobSchedule] - A list of PipelineJobSchedule resource objects.

list_jobs(filter: Optional[str] = None, order_by: Optional[str] = None, enable_simple_view: bool = True, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

List all PipelineJob ‘s created by this PipelineJobSchedule.

Example usage:

pipeline_job_schedule.list_jobs(order_by=’create_time_desc’)

  • Parameters

    • filter (str) – Optional. An expression for filtering the results of the request. For field names both snake_case and camelCase are supported.

    • order_by (str) – Optional. A comma-separated list of fields to order by, sorted in ascending order. Use “desc” after a field name for descending. Supported fields: display_name, create_time, update_time

    • enable_simple_view (bool) – Optional. Whether to pass the read_mask parameter to the list call. Defaults to True if not provided. This will improve the performance of calling list(). However, the returned PipelineJob list will not include all fields for each PipelineJob. Setting this to True will exclude the following fields in your response: runtime_config, service_account, network, and some subfields of pipeline_spec and job_detail. The following fields will be included in each PipelineJob resource in your response: state, display_name, pipeline_spec.pipeline_info, create_time, start_time, end_time, update_time, labels, template_uri, template_metadata.version, job_detail.pipeline_run_context, job_detail.pipeline_context.

    • project (str) – Optional. Project to retrieve list from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to retrieve list from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to retrieve list. Overrides credentials set in aiplatform.init.

  • Returns

    List[PipelineJob] - A list of PipelineJob resource objects.

property max_concurrent_run_count(: [int](https://python.readthedocs.io/en/latest/library/functions.html#int )

Current Schedule max_concurrent_run_count.

  • Returns

    Schedule max_concurrent_run_count.

property max_run_count(: [int](https://python.readthedocs.io/en/latest/library/functions.html#int )

Current Schedule max_run_count.

  • Returns

    Schedule max_run_count.

property name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Name of this resource.

pause()

Starts asynchronous pause on the Schedule.

Changes Schedule state from State.ACTIVE to State.PAUSED.

property resource_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Full qualified resource name.

resume(catch_up: bool = True)

Starts asynchronous resume on the Schedule.

Changes Schedule state from State.PAUSED to State.ACTIVE.

  • Parameters

    catch_up (bool) – Optional. Whether to backfill missed runs when the Schedule is resumed from State.PAUSED.

property state(: Optional[[google.cloud.aiplatform_v1.types.schedule.Schedule.State](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.Schedule.State)_ )

Current Schedule state.

  • Returns

    Schedule state.

to_dict()

Returns the resource proto as a dictionary.

update(display_name: Optional[str] = None, cron: Optional[str] = None, start_time: Optional[str] = None, end_time: Optional[str] = None, allow_queueing: Optional[bool] = None, max_run_count: Optional[int] = None, max_concurrent_run_count: Optional[int] = None)

Update an existing PipelineJobSchedule.

Example usage:

pipeline_job_schedule.update(

display_name=’updated-display-name’,
cron=’\* \* \* \* 

```python
*
```

’,

)

  • Parameters

    • display_name (str) – Optional. The user-defined name of this PipelineJobSchedule.

    • cron (str) – Optional. Time specification (cron schedule expression) to launch scheduled runs. To explicitly set a timezone to the cron tab, apply a prefix: “CRON_TZ=${IANA_TIME_ZONE}” or “TZ=${IANA_TIME_ZONE}”. The ${IANA_TIME_ZONE} may only be a valid string from IANA time zone database. For example, “CRON_TZ=America/New_York 1 * * * 

      *

      ”, or “TZ=America/New_York 1 * * * 

      *

      ”.

    • start_time (str) – Optional. Timestamp after which the first run can be scheduled. If unspecified, it defaults to the schedule creation timestamp.

    • end_time (str) – Optional. Timestamp after which no more runs will be scheduled. If unspecified, then runs will be scheduled indefinitely.

    • allow_queueing (bool) – Optional. Whether new scheduled runs can be queued when max_concurrent_runs limit is reached.

    • max_run_count (int) – Optional. Maximum run count of the schedule. If specified, The schedule will be completed when either started_run_count >= max_run_count or when end_time is reached. Must be positive and <= 2^63-1.

    • max_concurrent_run_count (int) – Optional. Maximum number of runs that can be started concurrently for this PipelineJobSchedule.

  • Raises

    RuntimeError – User tried to call update() before create().

property update_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was last updated.

wait()

Wait for all runs scheduled by this Schedule to complete.

class google.cloud.aiplatform.PrivateEndpoint(endpoint_name: str, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Bases: google.cloud.aiplatform.models.Endpoint

Represents a Vertex AI PrivateEndpoint resource.

Read more about private endpoints in the documentation.

Retrieves a PrivateEndpoint resource.

Example usage:

my_private_endpoint = aiplatform.PrivateEndpoint(

    endpoint_name=”projects/123/locations/us-central1/endpoints/1234567891234567890”

)

or (when project and location are initialized)

my_private_endpoint = aiplatform.PrivateEndpoint(

    endpoint_name=”1234567891234567890”

)

  • Parameters

    • endpoint_name (str) – Required. A fully-qualified endpoint resource name or endpoint ID. Example: “projects/123/locations/us-central1/endpoints/my_endpoint_id” or “my_endpoint_id” when project and location are initialized or passed.

    • project (str) – Optional. Project to retrieve endpoint from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to retrieve endpoint from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to upload this model. Overrides credentials set in aiplatform.init.

  • Raises

    • ValueError – If the Endpoint being retrieved is not a PrivateEndpoint.

    • ImportError – If there is an issue importing the urllib3 package.

client_class()

alias of google.cloud.aiplatform.utils.EndpointClientWithOverride

classmethod create(display_name: str, project: Optional[str] = None, location: Optional[str] = None, network: Optional[str] = None, description: Optional[str] = None, labels: Optional[Dict[str, str]] = None, credentials: Optional[google.auth.credentials.Credentials] = None, encryption_spec_key_name: Optional[str] = None, sync=True)

Creates a new PrivateEndpoint.

Example usage:

my_private_endpoint = aiplatform.PrivateEndpoint.create(

    display_name=”my_endpoint_name”,
    project=”my_project_id”,
    location=”us-central1”,
    network=”projects/123456789123/global/networks/my_vpc”

)

or (when project and location are initialized)

my_private_endpoint = aiplatform.PrivateEndpoint.create(

    display_name=”my_endpoint_name”,
    network=”projects/123456789123/global/networks/my_vpc”

)

  • Parameters

    • display_name (str) – Required. The user-defined name of the Endpoint. The name can be up to 128 characters long and can be consist of any UTF-8 characters.

    • project (str) – Optional. Project to retrieve endpoint from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to retrieve endpoint from. If not set, location set in aiplatform.init will be used.

    • network (str) – Optional. The full name of the Compute Engine network to which this Endpoint will be peered. E.g. “projects/123456789123/global/networks/my_vpc”. Private services access must already be configured for the network. If left unspecified, the network set with aiplatform.init will be used.

    • description (str) – Optional. The description of the Endpoint.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The labels with user-defined metadata to organize your Endpoints. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information and examples of labels.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to upload this model. Overrides credentials set in aiplatform.init.

    • encryption_spec_key_name (str) – Optional. The Cloud KMS resource identifier of the customer managed encryption key used to protect the model. Has the form: projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key. The key needs to be in the same region as where the compute resource is created.

      If set, this Model and all sub-resources of this Model will be secured by this key.

      Overrides encryption_spec_key_name set in aiplatform.init.

    • sync (bool) – Whether to execute this method synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

  • Returns

    Created endpoint.

  • Return type

    endpoint (aiplatform.PrivateEndpoint)

  • Raises

    ValueError – A network must be instantiated when creating a PrivateEndpoint.

property create_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was created.

delete(force: bool = False, sync: bool = True)

Deletes this Vertex AI PrivateEndpoint resource. If force is set to True, all models on this PrivateEndpoint will be undeployed prior to deletion.

  • Parameters

    • force (bool) – Required. If force is set to True, all deployed models on this Endpoint will be undeployed first. Default is False.

    • sync (bool) – Whether to execute this method synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

  • Raises

    FailedPrecondition – If models are deployed on this Endpoint and force = False.

deploy(model: google.cloud.aiplatform.models.Model, deployed_model_display_name: Optional[str] = None, machine_type: Optional[str] = None, min_replica_count: int = 1, max_replica_count: int = 1, accelerator_type: Optional[str] = None, accelerator_count: Optional[int] = None, tpu_topology: Optional[str] = None, service_account: Optional[str] = None, explanation_metadata: Optional[google.cloud.aiplatform_v1.types.explanation_metadata.ExplanationMetadata] = None, explanation_parameters: Optional[google.cloud.aiplatform_v1.types.explanation.ExplanationParameters] = None, metadata: Optional[Sequence[Tuple[str, str]]] = (), sync=True, disable_container_logging: bool = False)

Deploys a Model to the PrivateEndpoint.

Example Usage:

my_private_endpoint.deploy(

    model=my_model

)

  • Parameters

    • model (aiplatform.Model) – Required. Model to be deployed.

    • deployed_model_display_name (str) – Optional. The display name of the DeployedModel. If not provided upon creation, the Model’s display_name is used.

    • machine_type (str) – Optional. The type of machine. Not specifying machine type will result in model to be deployed with automatic resources.

    • min_replica_count (int) – Optional. The minimum number of machine replicas this deployed model will be always deployed on. If traffic against it increases, it may dynamically be deployed onto more replicas, and as traffic decreases, some of these extra replicas may be freed.

    • max_replica_count (int) – Optional. The maximum number of replicas this deployed model may be deployed on when the traffic against it increases. If requested value is too large, the deployment will error, but if deployment succeeds then the ability to scale the model to that many replicas is guaranteed (barring service outages). If traffic against the deployed model increases beyond what its replicas at maximum may handle, a portion of the traffic will be dropped. If this value is not provided, the larger value of min_replica_count or 1 will be used. If value provided is smaller than min_replica_count, it will automatically be increased to be min_replica_count.

    • accelerator_type (str) – Optional. Hardware accelerator type. Must also set accelerator_count if used. One of ACCELERATOR_TYPE_UNSPECIFIED, NVIDIA_TESLA_K80, NVIDIA_TESLA_P100, NVIDIA_TESLA_V100, NVIDIA_TESLA_P4, NVIDIA_TESLA_T4

    • accelerator_count (int) – Optional. The number of accelerators to attach to a worker replica.

    • tpu_topology (str) – Optional. The TPU topology to use for the DeployedModel. Required for CloudTPU multihost deployments.

    • service_account (str) – The service account that the DeployedModel’s container runs as. Specify the email address of the service account. If this service account is not specified, the container runs as a service account that doesn’t have access to the resource project. Users deploying the Model must have the iam.serviceAccounts.actAs permission on this service account.

    • explanation_metadata (aiplatform.explain.ExplanationMetadata) – Optional. Metadata describing the Model’s input and output for explanation. explanation_metadata is optional while explanation_parameters must be specified when used. For more details, see Ref docs http://tinyurl.com/1igh60kt

    • explanation_parameters (aiplatform.explain.ExplanationParameters) – Optional. Parameters to configure explaining for Model’s predictions. For more details, see Ref docs http://tinyurl.com/1an4zake

    • metadata (Sequence[Tuple[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*]) – Optional. Strings which should be sent along with the request as metadata.

    • sync (bool) – Whether to execute this method synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

property display_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Display name of this resource.

property encryption_spec(: Optional[[google.cloud.aiplatform_v1.types.encryption_spec.EncryptionSpec](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.EncryptionSpec)_ )

Customer-managed encryption key options for this Vertex AI resource.

If this is set, then all resources created by this Vertex AI resource will be encrypted with the provided encryption key.

explain()

Make a prediction with explanations against this Endpoint.

Example usage:

response = my_endpoint.explain(instances=[…])
my_explanations = response.explanations

  • Parameters

    • instances (List) – Required. The instances that are the input to the prediction call. A DeployedModel may have an upper limit on the number of instances it supports per request, and when it is exceeded the prediction call errors in case of AutoML Models, or, in case of customer created Models, the behaviour is as documented by that Model. The schema of any single instance may be specified via Endpoint’s DeployedModels’ [Model’s][google.cloud.aiplatform.v1beta1.DeployedModel.model] [PredictSchemata’s][google.cloud.aiplatform.v1beta1.Model.predict_schemata] instance_schema_uri.

    • parameters (Dict) – The parameters that govern the prediction. The schema of the parameters may be specified via Endpoint’s DeployedModels’ [Model’s ][google.cloud.aiplatform.v1beta1.DeployedModel.model] [PredictSchemata’s][google.cloud.aiplatform.v1beta1.Model.predict_schemata] parameters_schema_uri.

    • deployed_model_id (str) – Optional. If specified, this ExplainRequest will be served by the chosen DeployedModel, overriding this Endpoint’s traffic split.

    • timeout (float) – Optional. The timeout for this request in seconds.

  • Returns

    Prediction with returned predictions, explanations, and Model ID.

  • Return type

    prediction (aiplatform.Prediction)

async explain_async(instances: List[Dict], *, parameters: Optional[Dict] = None, deployed_model_id: Optional[str] = None, timeout: Optional[float] = None)

Make a prediction with explanations against this Endpoint.

Example usage:

`\`
response = await my_endpoint.explain_async(instances=[...])
my_explanations = response.explanations
\``

  • Parameters

    • instances (List) – Required. The instances that are the input to the prediction call. A DeployedModel may have an upper limit on the number of instances it supports per request, and when it is exceeded the prediction call errors in case of AutoML Models, or, in case of customer created Models, the behaviour is as documented by that Model. The schema of any single instance may be specified via Endpoint’s DeployedModels’ [Model’s][google.cloud.aiplatform.v1beta1.DeployedModel.model] [PredictSchemata’s][google.cloud.aiplatform.v1beta1.Model.predict_schemata] instance_schema_uri.

    • parameters (Dict) – The parameters that govern the prediction. The schema of the parameters may be specified via Endpoint’s DeployedModels’ [Model’s ][google.cloud.aiplatform.v1beta1.DeployedModel.model] [PredictSchemata’s][google.cloud.aiplatform.v1beta1.Model.predict_schemata] parameters_schema_uri.

    • deployed_model_id (str) – Optional. If specified, this ExplainRequest will be served by the chosen DeployedModel, overriding this Endpoint’s traffic split.

    • timeout (float) – Optional. The timeout for this request in seconds.

  • Returns

    Prediction with returned predictions, explanations, and Model ID.

  • Return type

    prediction (aiplatform.Prediction)

property explain_http_uri(: Optional[str )

HTTP path to send explain requests to, used when calling PrivateEndpoint.explain()

property gca_resource(: [proto.message.Message](https://proto-plus-python.readthedocs.io/en/latest/reference/message.html#proto.message.Message )

The underlying resource proto representation.

health_check()

Makes a request to this PrivateEndpoint’s health check URI. Must be within network that this PrivateEndpoint is in.

Example Usage:

if my_private_endpoint.health_check():

    print(“PrivateEndpoint is healthy!”)

  • Returns

    Checks if calls can be made to this PrivateEndpoint.

  • Return type

    bool

  • Raises

    RuntimeError – If a model has not been deployed a request cannot be made.

property health_http_uri(: Optional[str )

HTTP path to send health check requests to, used when calling PrivateEndpoint.health_check()

property labels(: Dict[str, str )

User-defined labels containing metadata about this resource.

Read more about labels at https://goo.gl/xmQnxf

classmethod list(filter: Optional[str] = None, order_by: Optional[str] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

List all PrivateEndpoint resource instances.

Example Usage:

my_private_endpoints = aiplatform.PrivateEndpoint.list()

or

my_private_endpoints = aiplatform.PrivateEndpoint.list(

    filter=’labels.my_label=”my_label_value” OR display_name=!”old_endpoint”’,

)

  • Parameters

    • filter (str) – Optional. An expression for filtering the results of the request. For field names both snake_case and camelCase are supported.

    • order_by (str) – Optional. A comma-separated list of fields to order by, sorted in ascending order. Use “desc” after a field name for descending. Supported fields: display_name, create_time, update_time

    • project (str) – Optional. Project to retrieve list from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to retrieve list from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to retrieve list. Overrides credentials set in aiplatform.init.

  • Returns

    A list of PrivateEndpoint resource objects.

  • Return type

    List[models.PrivateEndpoint]

list_models()

Returns a list of the models deployed to this Endpoint.

  • Returns

    A list of the models deployed in this Endpoint.

  • Return type

    deployed_models (List[aiplatform.gapic.DeployedModel])

property name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Name of this resource.

property network(: Optional[str )

The full name of the Google Compute Engine network to which this Endpoint should be peered.

Takes the format projects/{project}/global/networks/{network}. Where {project} is a project number, as in 12345, and {network} is a network name.

Private services access must already be configured for the network. If left unspecified, the Endpoint is not peered with any network.

predict(instances: List, parameters: Optional[Dict] = None)

Make a prediction against this PrivateEndpoint using a HTTP request. This method must be called within the network the PrivateEndpoint is peered to. Otherwise, the predict() call will fail with error code 404. To check, use PrivateEndpoint.network.

Example usage:

response = my_private_endpoint.predict(instances=[…])
my_predictions = response.predictions

  • Parameters

    • instances (List) – Required. The instances that are the input to the prediction call. Instance types mut be JSON serializable. A DeployedModel may have an upper limit on the number of instances it supports per request, and when it is exceeded the prediction call errors in case of AutoML Models, or, in case of customer created Models, the behaviour is as documented by that Model. The schema of any single instance may be specified via Endpoint’s DeployedModels’ [Model’s][google.cloud.aiplatform.v1beta1.DeployedModel.model] [PredictSchemata’s][google.cloud.aiplatform.v1beta1.Model.predict_schemata] instance_schema_uri.

    • parameters (Dict) – The parameters that govern the prediction. The schema of the parameters may be specified via Endpoint’s DeployedModels’ [Model’s ][google.cloud.aiplatform.v1beta1.DeployedModel.model] [PredictSchemata’s][google.cloud.aiplatform.v1beta1.Model.predict_schemata] parameters_schema_uri.

  • Returns

    Prediction object with returned predictions and Model ID.

  • Return type

    prediction (aiplatform.Prediction)

  • Raises

    RuntimeError – If a model has not been deployed a request cannot be made.

async predict_async(instances: List, *, parameters: Optional[Dict] = None, timeout: Optional[float] = None)

Make an asynchronous prediction against this Endpoint. Example usage:

\ response = await my_endpoint.predict_async(instances=[...]) my_predictions = response.predictions ``

  • Parameters

    • instances (List) – Required. The instances that are the input to the prediction call. A DeployedModel may have an upper limit on the number of instances it supports per request, and when it is exceeded the prediction call errors in case of AutoML Models, or, in case of customer created Models, the behaviour is as documented by that Model. The schema of any single instance may be specified via Endpoint’s DeployedModels’ [Model’s][google.cloud.aiplatform.v1beta1.DeployedModel.model] [PredictSchemata’s][google.cloud.aiplatform.v1beta1.Model.predict_schemata] instance_schema_uri.

    • parameters (Dict) – Optional. The parameters that govern the prediction. The schema of the parameters may be specified via Endpoint’s DeployedModels’ [Model’s ][google.cloud.aiplatform.v1beta1.DeployedModel.model] [PredictSchemata’s][google.cloud.aiplatform.v1beta1.Model.predict_schemata] parameters_schema_uri.

    • timeout (float) – Optional. The timeout for this request in seconds.

  • Returns

    Prediction with returned predictions and Model ID.

  • Return type

    prediction (aiplatform.Prediction)

property predict_http_uri(: Optional[str )

HTTP path to send prediction requests to, used when calling PrivateEndpoint.predict()

property preview()

Return an Endpoint instance with preview features enabled.

raw_predict(body: bytes, headers: Dict[str, str])

Make a prediction request using arbitrary headers. This method must be called within the network the PrivateEndpoint is peered to. Otherwise, the predict() call will fail with error code 404. To check, use PrivateEndpoint.network.

Example usage:

my_endpoint = aiplatform.PrivateEndpoint(ENDPOINT_ID)
response = my_endpoint.raw_predict(

> body = b’{“instances”:[{“feat_1”:val_1, “feat_2”:val_2}]}’
> headers = {‘Content-Type’:’application/json’}

)
status_code = response.status_code
results = json.dumps(response.text)

  • Parameters

  • Returns

    A requests.models.Response object containing the status code and prediction results.

property resource_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Full qualified resource name.

to_dict()

Returns the resource proto as a dictionary.

property traffic_split(: Dict[str, int )

A map from a DeployedModel’s ID to the percentage of this Endpoint’s traffic that should be forwarded to that DeployedModel.

If a DeployedModel’s ID is not listed in this map, then it receives no traffic.

The traffic percentage values must add up to 100, or map must be empty if the Endpoint is to not accept any traffic at a moment.

undeploy(deployed_model_id: str, sync=True)

Undeploys a deployed model from the PrivateEndpoint.

Example Usage:

my_private_endpoint.undeploy(

    deployed_model_id=”1234567891232567891”

)

or

my_deployed_model_id = my_private_endpoint.list_models()[0].id
my_private_endpoint.undeploy(

> deployed_model_id=my_deployed_model_id

)

  • Parameters

    • deployed_model_id (str) – Required. The ID of the DeployedModel to be undeployed from the PrivateEndpoint. Use PrivateEndpoint.list_models() to get the deployed model ID.

    • sync (bool) – Whether to execute this method synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

undeploy_all(sync: bool = True)

Undeploys every model deployed to this Endpoint.

  • Parameters

    sync (bool) – Whether to execute this method synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

update(display_name: Optional[str] = None, description: Optional[str] = None, labels: Optional[Dict[str, str]] = None, traffic_split: Optional[Dict[str, int]] = None, request_metadata: Optional[Sequence[Tuple[str, str]]] = (), update_request_timeout: Optional[float] = None)

Updates an endpoint.

Example usage:

my_endpoint = my_endpoint.update(

    display_name=’my-updated-endpoint’,
    description=’my updated description’,
    labels={‘key’: ‘value’},
    traffic_split={

    > ‘123456’: 20,
    > ‘234567’: 80,

    },

)

  • Parameters

    • display_name (str) – Optional. The display name of the Endpoint. The name can be up to 128 characters long and can be consist of any UTF-8 characters.

    • description (str) – Optional. The description of the Endpoint.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The labels with user-defined metadata to organize your Endpoints. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information and examples of labels.

    • traffic_split (Dict[str, *[int](https://python.readthedocs.io/en/latest/library/functions.html#int)]*) – Optional. A map from a DeployedModel’s ID to the percentage of this Endpoint’s traffic that should be forwarded to that DeployedModel. If a DeployedModel’s ID is not listed in this map, then it receives no traffic. The traffic percentage values must add up to 100, or map must be empty if the Endpoint is to not accept any traffic at a moment.

    • request_metadata (Sequence[Tuple[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*]) – Optional. Strings which should be sent along with the request as metadata.

    • update_request_timeout (float) – Optional. The timeout for the update request in seconds.

  • Returns

    Updated endpoint resource.

  • Return type

    Endpoint (aiplatform.Prediction)

  • Raises

    ValueError – If labels is not the correct format.

property update_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was last updated.

wait()

Helper method that blocks until all futures are complete.

class google.cloud.aiplatform.SequenceToSequencePlusForecastingTrainingJob(display_name: Optional[str] = None, optimization_objective: Optional[str] = None, column_specs: Optional[Dict[str, str]] = None, column_transformations: Optional[List[Dict[str, Dict[str, str]]]] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None, labels: Optional[Dict[str, str]] = None, training_encryption_spec_key_name: Optional[str] = None, model_encryption_spec_key_name: Optional[str] = None)

Bases: google.cloud.aiplatform.training_jobs._ForecastingTrainingJob

Class to train Sequence to Sequence (Seq2Seq) forecasting models.

Constructs a Forecasting Training Job.

  • Parameters

    • display_name (str) – Optional. The user-defined name of this TrainingPipeline.

    • optimization_objective (str) – Optional. Objective function the model is to be optimized towards. The training process creates a Model that optimizes the value of the objective function over the validation set. The supported optimization objectives: “minimize-rmse” (default) - Minimize root-mean-squared error (RMSE). “minimize-mae” - Minimize mean-absolute error (MAE). “minimize-rmsle” - Minimize root-mean-squared log error (RMSLE). “minimize-rmspe” - Minimize root-mean-squared percentage error (RMSPE). “minimize-wape-mae” - Minimize the combination of weighted absolute percentage error (WAPE)

      and mean-absolute-error (MAE).

      ”minimize-quantile-loss” - Minimize the quantile loss at the defined quantiles.

      (Set this objective to build quantile forecasts.)

    • column_specs (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. Alternative to column_transformations where the keys of the dict are column names and their respective values are one of AutoMLTabularTrainingJob.column_data_types. When creating transformation for BigQuery Struct column, the column should be flattened using “.” as the delimiter. Only columns with no child should have a transformation. If an input column has no transformations on it, such a column is ignored by the training, except for the targetColumn, which should have no transformations defined on. Only one of column_transformations or column_specs should be passed.

    • column_transformations (List[Dict[str, **Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]]]*) – Optional. Transformations to apply to the input columns (i.e. columns other than the targetColumn). Each transformation may produce multiple result values from the column’s value, and all are used for training. When creating transformation for BigQuery Struct column, the column should be flattened using “.” as the delimiter. Only columns with no child should have a transformation. If an input column has no transformations on it, such a column is ignored by the training, except for the targetColumn, which should have no transformations defined on. Only one of column_transformations or column_specs should be passed. Consider using column_specs as column_transformations will be deprecated eventually.

    • project (str) – Optional. Project to run training in. Overrides project set in aiplatform.init.

    • location (str) – Optional. Location to run training in. Overrides location set in aiplatform.init.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to run call training service. Overrides credentials set in aiplatform.init.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The labels with user-defined metadata to organize TrainingPipelines. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information and examples of labels.

    • training_encryption_spec_key_name (Optional[str]) – Optional. The Cloud KMS resource identifier of the customer managed encryption key used to protect the training pipeline. Has the form: projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key. The key needs to be in the same region as where the compute resource is created. If set, this TrainingPipeline will be secured by this key. Note: Model trained by this TrainingPipeline is also secured by this key if model_to_upload is not set separately. Overrides encryption_spec_key_name set in aiplatform.init.

    • model_encryption_spec_key_name (Optional[str]) – Optional. The Cloud KMS resource identifier of the customer managed encryption key used to protect the model. Has the form: projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key. The key needs to be in the same region as where the compute resource is created. If set, the trained Model will be secured by this key. Overrides encryption_spec_key_name set in aiplatform.init.

  • Raises

    ValueError – If both column_transformations and column_specs were provided.

cancel()

Starts asynchronous cancellation on the TrainingJob. The server makes a best effort to cancel the job, but success is not guaranteed. On successful cancellation, the TrainingJob is not deleted; instead it becomes a job with state set to CANCELLED.

  • Raises

    RuntimeError – If this TrainingJob has not started running.

client_class()

alias of google.cloud.aiplatform.utils.PipelineClientWithOverride

property create_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was created.

delete(sync: bool = True)

Deletes this Vertex AI resource. WARNING: This deletion is permanent.

  • Parameters

    sync (bool) – Whether to execute this deletion synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

property display_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Display name of this resource.

done()

Method indicating whether a job has completed.

  • Returns

    True if the job has completed.

property encryption_spec(: Optional[[google.cloud.aiplatform_v1.types.encryption_spec.EncryptionSpec](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.EncryptionSpec)_ )

Customer-managed encryption key options for this Vertex AI resource.

If this is set, then all resources created by this Vertex AI resource will be encrypted with the provided encryption key.

property end_time(: Optional[datetime.datetime )

Time when the TrainingJob resource entered the PIPELINE_STATE_SUCCEEDED, PIPELINE_STATE_FAILED, PIPELINE_STATE_CANCELLED state.

property error(: Optional[google.rpc.status_pb2.Status )

Detailed error info for this TrainingJob resource. Only populated when the TrainingJob’s state is PIPELINE_STATE_FAILED or PIPELINE_STATE_CANCELLED.

property evaluated_data_items_bigquery_uri(: Optional[str )

BigQuery location of exported evaluated examples from the Training Job :returns:

BigQuery uri for the exported evaluated examples if the export

feature is enabled for training.

None: If the export feature was not enabled for training.

  • Return type

    str

property gca_resource(: [proto.message.Message](https://proto-plus-python.readthedocs.io/en/latest/reference/message.html#proto.message.Message )

The underlying resource proto representation.

classmethod get(resource_name: str, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Get Training Job for the given resource_name.

  • Parameters

    • resource_name (str) – Required. A fully-qualified resource name or ID.

    • project (str) – Optional project to retrieve training job from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional location to retrieve training job from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Custom credentials to use to upload this model. Overrides credentials set in aiplatform.init.

  • Raises

    ValueError – If the retrieved training job’s training task definition doesn’t match the custom training task definition.

  • Returns

    A Vertex AI Training Job

get_model(sync=True)

Vertex AI Model produced by this training, if one was produced.

  • Parameters

    sync (bool) – Whether to execute this method synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

  • Returns

    Vertex AI Model produced by this training

  • Return type

    model

  • Raises

    RuntimeError – If training failed or if a model was not produced by this training.

property has_failed(: [bool](https://python.readthedocs.io/en/latest/library/functions.html#bool )

Returns True if training has failed.

False otherwise.

property labels(: Dict[str, str )

User-defined labels containing metadata about this resource.

Read more about labels at https://goo.gl/xmQnxf

classmethod list(filter: Optional[str] = None, order_by: Optional[str] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

List all instances of this TrainingJob resource.

Example Usage:

aiplatform.CustomTrainingJob.list(

filter=’display_name=”experiment_a27”’,
order_by=’create_time desc’

)

  • Parameters

    • filter (str) – Optional. An expression for filtering the results of the request. For field names both snake_case and camelCase are supported.

    • order_by (str) – Optional. A comma-separated list of fields to order by, sorted in ascending order. Use “desc” after a field name for descending. Supported fields: display_name, create_time, update_time

    • project (str) – Optional. Project to retrieve list from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to retrieve list from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to retrieve list. Overrides credentials set in aiplatform.init.

  • Returns

    List[VertexAiResourceNoun] - A list of TrainingJob resource objects

property name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Name of this resource.

property resource_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Full qualified resource name.

run(dataset: google.cloud.aiplatform.datasets.time_series_dataset.TimeSeriesDataset, target_column: str, time_column: str, time_series_identifier_column: str, unavailable_at_forecast_columns: List[str], available_at_forecast_columns: List[str], forecast_horizon: int, data_granularity_unit: str, data_granularity_count: int, training_fraction_split: Optional[float] = None, validation_fraction_split: Optional[float] = None, test_fraction_split: Optional[float] = None, predefined_split_column_name: Optional[str] = None, timestamp_split_column_name: Optional[str] = None, weight_column: Optional[str] = None, time_series_attribute_columns: Optional[List[str]] = None, context_window: Optional[int] = None, export_evaluated_data_items: bool = False, export_evaluated_data_items_bigquery_destination_uri: Optional[str] = None, export_evaluated_data_items_override_destination: bool = False, quantiles: Optional[List[float]] = None, validation_options: Optional[str] = None, budget_milli_node_hours: int = 1000, model_display_name: Optional[str] = None, model_labels: Optional[Dict[str, str]] = None, model_id: Optional[str] = None, parent_model: Optional[str] = None, is_default_version: Optional[bool] = True, model_version_aliases: Optional[Sequence[str]] = None, model_version_description: Optional[str] = None, additional_experiments: Optional[List[str]] = None, hierarchy_group_columns: Optional[List[str]] = None, hierarchy_group_total_weight: Optional[float] = None, hierarchy_temporal_total_weight: Optional[float] = None, hierarchy_group_temporal_total_weight: Optional[float] = None, window_column: Optional[str] = None, window_stride_length: Optional[int] = None, window_max_count: Optional[int] = None, holiday_regions: Optional[List[str]] = None, sync: bool = True, create_request_timeout: Optional[float] = None, enable_probabilistic_inference: bool = False)

Runs the training job and returns a model.

If training on a Vertex AI dataset, you can use one of the following split configurations:

Data fraction splits:
Any of `training_fraction_split`, `validation_fraction_split` and
`test_fraction_split` may optionally be provided, they must sum to up to 1. If
the provided ones sum to less than 1, the remainder is assigned to sets as
decided by Vertex AI. If none of the fractions are set, by default roughly 80%
of data will be used for training, 10% for validation, and 10% for test.

Predefined splits:
Assigns input data to training, validation, and test sets based on the value of a provided key.
If using predefined splits, `predefined_split_column_name` must be provided.
Supported only for tabular Datasets.

Timestamp splits:
Assigns input data to training, validation, and test sets
based on a provided timestamps. The youngest data pieces are
assigned to training set, next to validation set, and the oldest
to the test set.
Supported only for tabular Datasets.

  • Parameters

    • dataset (datasets.TimeSeriesDataset) – Required. The dataset within the same Project from which data will be used to train the Model. The Dataset must use schema compatible with Model being trained, and what is compatible should be described in the used TrainingPipeline’s [training_task_definition] [google.cloud.aiplatform.v1beta1.TrainingPipeline.training_task_definition]. For time series Datasets, all their data is exported to training, to pick and choose from.

    • target_column (str) – Required. Name of the column that the Model is to predict values for. This column must be unavailable at forecast.

    • time_column (str) – Required. Name of the column that identifies time order in the time series. This column must be available at forecast.

    • time_series_identifier_column (str) – Required. Name of the column that identifies the time series.

    • unavailable_at_forecast_columns (List[str]) – Required. Column names of columns that are unavailable at forecast. Each column contains information for the given entity (identified by the [time_series_identifier_column]) that is unknown before the forecast (e.g. population of a city in a given year, or weather on a given day).

    • available_at_forecast_columns (List[str]) – Required. Column names of columns that are available at forecast. Each column contains information for the given entity (identified by the [time_series_identifier_column]) that is known at forecast.

    • forecast_horizon – (int): Required. The amount of time into the future for which forecasted values for the target are returned. Expressed in number of units defined by the [data_granularity_unit] and [data_granularity_count] field. Inclusive.

    • data_granularity_unit (str) – Required. The data granularity unit. Accepted values are minute, hour, day, week, month, year.

    • data_granularity_count (int) – Required. The number of data granularity units between data points in the training data. If [data_granularity_unit] is minute, can be 1, 5, 10, 15, or 30. For all other values of [data_granularity_unit], must be 1.

    • predefined_split_column_name (str) – Optional. The key is a name of one of the Dataset’s data columns. The value of the key (either the label’s value or value in the column) must be one of {TRAIN, VALIDATE, TEST}, and it defines to which set the given piece of data is assigned. If for a piece of data the key is not present or has an invalid value, that piece is ignored by the pipeline.

      Supported only for tabular and time series Datasets.

    • timestamp_split_column_name (str) – Optional. The key is a name of one of the Dataset’s data columns. The value of the key values of the key (the values in the column) must be in RFC 3339 date-time format, where time-offset = “Z” (e.g. 1985-04-12T23:20:50.52Z). If for a piece of data the key is not present or has an invalid value, that piece is ignored by the pipeline. Supported only for tabular and time series Datasets. This parameter must be used with training_fraction_split, validation_fraction_split, and test_fraction_split.

    • weight_column (str) – Optional. Name of the column that should be used as the weight column. Higher values in this column give more importance to the row during Model training. The column must have numeric values between 0 and 10000 inclusively, and 0 value means that the row is ignored. If the weight column field is not set, then all rows are assumed to have equal weight of 1.

    • time_series_attribute_columns (List[str]) – Optional. Column names that should be used as attribute columns. Each column is constant within a time series.

    • context_window (int) – Optional. The amount of time into the past training and prediction data is used for model training and prediction respectively. Expressed in number of units defined by the [data_granularity_unit] and [data_granularity_count] fields. When not provided uses the default value of 0 which means the model sets each series context window to be 0 (also known as “cold start”). Inclusive.

    • export_evaluated_data_items (bool) – Whether to export the test set predictions to a BigQuery table. If False, then the export is not performed.

    • export_evaluated_data_items_bigquery_destination_uri (string) – Optional. URI of desired destination BigQuery table for exported test set predictions.

      Expected format: bq://<project_id>:<dataset_id>:<table>

      If not specified, then results are exported to the following auto-created BigQuery table: <project_id>:export_evaluated_examples_<model_name>_<yyyy_MM_dd'T'HH_mm_ss_SSS'Z'>.evaluated_examples

      Applies only if [export_evaluated_data_items] is True.

    • export_evaluated_data_items_override_destination (bool) – Whether to override the contents of [export_evaluated_data_items_bigquery_destination_uri], if the table exists, for exported test set predictions. If False, and the table exists, then the training job will fail.

      Applies only if [export_evaluated_data_items] is True and [export_evaluated_data_items_bigquery_destination_uri] is specified.

    • quantiles (List[float]) – Quantiles to use for the minimize-quantile-loss [AutoMLForecastingTrainingJob.optimization_objective]. This argument is required in this case.

      Accepts up to 5 quantiles in the form of a double from 0 to 1, exclusive. Each quantile must be unique.

    • validation_options (str) – Validation options for the data validation component. The available options are: “fail-pipeline” - (default), will validate against the validation and fail the pipeline

      if it fails.

      ”ignore-validation” - ignore the results of the validation and continue the pipeline

    • budget_milli_node_hours (int) – Optional. The train budget of creating this Model, expressed in milli node hours i.e. 1,000 value in this field means 1 node hour. The training cost of the model will not exceed this budget. The final cost will be attempted to be close to the budget, though may end up being (even) noticeably smaller - at the backend’s discretion. This especially may happen when further model training ceases to provide any improvements. If the budget is set to a value known to be insufficient to train a Model for the given training set, the training won’t be attempted and will error. The minimum value is 1000 and the maximum is 72000.

    • model_display_name (str) – Optional. If the script produces a managed Vertex AI Model. The display name of the Model. The name can be up to 128 characters long and can be consist of any UTF-8 characters.

      If not provided upon creation, the job’s display_name is used.

    • model_labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The labels with user-defined metadata to organize your Models. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information and examples of labels.

    • model_id (str) – Optional. The ID to use for the Model produced by this job, which will become the final component of the model resource name. This value may be up to 63 characters, and valid characters are [a-z0-9_-]. The first character cannot be a number or hyphen.

    • parent_model (str) – Optional. The resource name or model ID of an existing model. The new model uploaded by this job will be a version of parent_model.

      Only set this field when training a new version of an existing model.

    • is_default_version (bool) – Optional. When set to True, the newly uploaded model version will automatically have alias “default” included. Subsequent uses of the model produced by this job without a version specified will use this “default” version.

      When set to False, the “default” alias will not be moved. Actions targeting the model version produced by this job will need to specifically reference this version by ID or alias.

      New model uploads, i.e. version 1, will always be “default” aliased.

    • model_version_aliases (Sequence[str]) – Optional. User provided version aliases so that the model version uploaded by this job can be referenced via alias instead of auto-generated version ID. A default version alias will be created for the first version of the model.

      The format is [a-z][a-zA-Z0-9-]{0,126}[a-z0-9]

    • model_version_description (str) – Optional. The description of the model version being uploaded by this job.

    • additional_experiments (List[str]) – Optional. Additional experiment flags for the time series forcasting training.

    • create_request_timeout (float) – Optional. The timeout for the create request in seconds.

    • hierarchy_group_columns (List[str]) – Optional. A list of time series attribute column names that define the time series hierarchy. Only one level of hierarchy is supported, ex. region for a hierarchy of stores or department for a hierarchy of products. If multiple columns are specified, time series will be grouped by their combined values, ex. (blue, large) for color and size, up to 5 columns are accepted. If no group columns are specified, all time series are considered to be part of the same group.

    • hierarchy_group_total_weight (float) – Optional. The weight of the loss for predictions aggregated over time series in the same hierarchy group.

    • hierarchy_temporal_total_weight (float) – Optional. The weight of the loss for predictions aggregated over the horizon for a single time series.

    • hierarchy_group_temporal_total_weight (float) – Optional. The weight of the loss for predictions aggregated over both the horizon and time series in the same hierarchy group.

    • window_column (str) – Optional. Name of the column that should be used to filter input rows. The column should contain either booleans or string booleans; if the value of the row is True, generate a sliding window from that row.

    • window_stride_length (int) – Optional. Step length used to generate input examples. Every window_stride_length rows will be used to generate a sliding window.

    • window_max_count (int) – Optional. Number of rows that should be used to generate input examples. If the total row count is larger than this number, the input data will be randomly sampled to hit the count.

    • holiday_regions (List[str]) – Optional. The geographical regions to use when creating holiday features. This option is only allowed when data_granularity_unit is day. Acceptable values can come from any of the following levels:

      Top level: GLOBAL Second level: continental regions

      NA: North America JAPAC: Japan and Asia Pacific EMEA: Europe, the Middle East and Africa LAC: Latin America and the Caribbean

      Third level: countries from ISO 3166-1 Country codes.

    • sync (bool) – Whether to execute this method synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

    • enable_probabilistic_inference (bool) – If probabilistic inference is enabled, the model will fit a distribution that captures the uncertainty of a prediction. At inference time, the predictive distribution is used to make a point prediction that minimizes the optimization objective. For example, the mean of a predictive distribution is the point prediction that minimizes RMSE loss. If quantiles are specified, then the quantiles of the distribution are also returned. The optimization objective cannot be minimize-quantile-loss.

  • Returns

    The trained Vertex AI Model resource or None if training did not

      produce a Vertex AI Model.

  • Return type

    model

  • Raises

    RuntimeError – If Training job has already been run or is waiting to run.

property start_time(: Optional[datetime.datetime )

Time when the TrainingJob entered the PIPELINE_STATE_RUNNING for the first time.

property state(: Optional[[google.cloud.aiplatform_v1.types.pipeline_state.PipelineState](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.PipelineState)_ )

Current training state.

to_dict()

Returns the resource proto as a dictionary.

property update_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was last updated.

wait()

Helper method that blocks until all futures are complete.

wait_for_resource_creation()

Waits until resource has been created.

class google.cloud.aiplatform.TabularDataset(dataset_name: str, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Bases: google.cloud.aiplatform.datasets.column_names_dataset._ColumnNamesDataset

A managed tabular dataset resource for Vertex AI.

Use this class to work with tabular datasets. You can use a CSV file, BigQuery, or a pandas DataFrame to create a tabular dataset. For more information about paging through BigQuery data, see Read data with BigQuery API using pagination. For more information about tabular data, see Tabular data.

The following code shows you how to create and import a tabular dataset with a CSV file.

``

`

py my_dataset = aiplatform.TabularDataset.create(

display_name=”my-dataset”, gcs_source=[‘gs://path/to/my/dataset.csv’])

``

`

The following code shows you how to create and import a tabular dataset in two distinct steps.

``

`

py my_dataset = aiplatform.TextDataset.create(

display_name=”my-dataset”)

my_dataset.import(

gcs_source=[‘gs://path/to/my/dataset.csv’]
import_schema_uri=aiplatform.schema.dataset.ioformat.text.multi_label_classification

If you create a tabular dataset with a pandas DataFrame, you need to use a BigQuery table to stage the data for Vertex AI:

``

`

py my_dataset = aiplatform.TabularDataset.create_from_dataframe(

df_source=my_pandas_dataframe, staging_path=f”bq://{bq_dataset_id}.table-unique”

Retrieves an existing managed dataset given a dataset name or ID.

  • Parameters

    • dataset_name (str) – Required. A fully-qualified dataset resource name or dataset ID. Example: “projects/123/locations/us-central1/datasets/456” or “456” when project and location are initialized or passed.

    • project (str) – Optional project to retrieve dataset from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional location to retrieve dataset from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Custom credentials to use to retrieve this Dataset. Overrides credentials set in aiplatform.init.

client_class()

alias of google.cloud.aiplatform.utils.DatasetClientWithOverride

property column_names(: List[str )

Retrieve the columns for the dataset by extracting it from the Google Cloud Storage or Google BigQuery source.

  • Returns

    List[str]

      A list of columns names

  • Raises

    RuntimeError – When no valid source is found.

classmethod create(display_name: Optional[str] = None, gcs_source: Optional[Union[str, Sequence[str]]] = None, bq_source: Optional[str] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None, request_metadata: Optional[Sequence[Tuple[str, str]]] = (), labels: Optional[Dict[str, str]] = None, encryption_spec_key_name: Optional[str] = None, sync: bool = True, create_request_timeout: Optional[float] = None)

Creates a tabular dataset.

  • Parameters

    • display_name (str) – Optional. The user-defined name of the dataset. The name must contain 128 or fewer UTF-8 characters.

    • gcs_source (Union[str, **Sequence[str]]) – Optional. The URI to one or more Google Cloud Storage buckets that contain your datasets. For example, str: “gs://bucket/file.csv” or Sequence[str]: [“gs://bucket/file1.csv”, “gs://bucket/file2.csv”].

    • bq_source (str) – Optional. The URI to a BigQuery table that’s used as an input source. For example, bq://project.dataset.table_name.

    • project (str) – Optional. The name of the Google Cloud project to which this TabularDataset is uploaded. This overrides the project that was set by aiplatform.init.

    • location (str) – Optional. The Google Cloud region where this dataset is uploaded. This region overrides the region that was set by aiplatform.init.

    • credentials (auth_credentials.Credentials) – Optional. The credentials that are used to upload the TabularDataset. These credentials override the credentials set by aiplatform.init.

    • request_metadata (Sequence[Tuple[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*]) – Optional. Strings that contain metadata that’s sent with the request.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. Labels with user-defined metadata to organize your Vertex AI Tensorboards. The maximum length of a key and of a value is 64 unicode characters. Labels and keys can contain only lowercase letters, numeric characters, underscores, and dashes. International characters are allowed. No more than 64 user labels can be associated with one Tensorboard (system labels are excluded). For more information and examples of using labels, see Using labels to organize Google Cloud Platform resources. System reserved label keys are prefixed with aiplatform.googleapis.com/ and are immutable.

    • encryption_spec_key_name (Optional[str]) – Optional. The Cloud KMS resource identifier of the customer managed encryption key that’s used to protect the dataset. The format of the key is projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key. The key needs to be in the same region as where the compute resource is created.

      If encryption_spec_key_name is set, this TabularDataset and all of its sub-resources are secured by this key.

      This encryption_spec_key_name overrides the encryption_spec_key_name set by aiplatform.init.

    • sync (bool) – If true, the create method creates a tabular dataset synchronously. If false, the create method creates a tabular dataset asynchronously.

    • create_request_timeout (float) – Optional. The number of seconds for the timeout of the create request.

  • Returns

    An instantiated representation of the managed TabularDataset resource.

  • Return type

    tabular_dataset (TabularDataset)

classmethod create_from_dataframe(df_source: pd.DataFrame, staging_path: str, bq_schema: Optional[Union[str, google.cloud.bigquery.schema.SchemaField]] = None, display_name: Optional[str] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Creates a new tabular dataset from a pandas DataFrame.

  • Parameters

    • df_source (pd.DataFrame) – Required. A pandas DataFrame containing the source data for ingestion as a TabularDataset. This method uses the data types from the provided DataFrame when the TabularDataset is created.

    • staging_path (str) – Required. The BigQuery table used to stage the data for Vertex AI. Because Vertex AI maintains a reference to this source to create the TabularDataset, you shouldn’t delete this BigQuery table. For example: bq://my-project.my-dataset.my-table. If the specified BigQuery table doesn’t exist, then the table is created for you. If the provided BigQuery table already exists, and the schemas of the BigQuery table and your DataFrame match, then the data in your local DataFrame is appended to the table. The location of the BigQuery table must conform to the BigQuery location requirements.

    • bq_schema (Optional[Union[str, **bigquery.SchemaField]]) – Optional. If not set, BigQuery autodetects the schema using the column types of your DataFrame. If set, BigQuery uses the schema you provide when the staging table is created. For more information, see the BigQuery LoadJobConfig.schema property.

    • display_name (str) – Optional. The user-defined name of the Dataset. The name must contain 128 or fewer UTF-8 characters.

    • project (str) – Optional. The project to upload this dataset to. This overrides the project set using aiplatform.init.

    • location (str) – Optional. The location to upload this dataset to. This overrides the location set using aiplatform.init.

    • credentials (auth_credentials.Credentials) – Optional. The custom credentials used to upload this dataset. This overrides credentials set using aiplatform.init.

  • Returns

    An instantiated representation of the managed TabularDataset resource.

  • Return type

    tabular_dataset (TabularDataset)

property create_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was created.

delete(sync: bool = True)

Deletes this Vertex AI resource. WARNING: This deletion is permanent.

  • Parameters

    sync (bool) – Whether to execute this deletion synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

property display_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Display name of this resource.

property encryption_spec(: Optional[[google.cloud.aiplatform_v1.types.encryption_spec.EncryptionSpec](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.EncryptionSpec)_ )

Customer-managed encryption key options for this Vertex AI resource.

If this is set, then all resources created by this Vertex AI resource will be encrypted with the provided encryption key.

export_data(output_dir: str)

Exports data to output dir to GCS.

  • Parameters

    output_dir (str) – Required. The Google Cloud Storage location where the output is to be written to. In the given directory a new directory will be created with name: export-data-<dataset-display-name>-<timestamp-of-export-call> where timestamp is in YYYYMMDDHHMMSS format. All export output will be written into that directory. Inside that directory, annotations with the same schema will be grouped into sub directories which are named with the corresponding annotations’ schema title. Inside these sub directories, a schema.yaml will be created to describe the output format.

    If the uri doesn’t end with ‘/’, a ‘/’ will be automatically appended. The directory is created if it doesn’t exist.

  • Returns

    All of the files that are exported in this export operation.

  • Return type

    exported_files (Sequence[str])

export_data_for_custom_training(output_dir: str, annotation_filter: Optional[str] = None, saved_query_id: Optional[str] = None, annotation_schema_uri: Optional[str] = None, split: Optional[Union[Dict[str, str], Dict[str, float]]] = None)

Exports data to output dir to GCS for custom training use case.

Example annotation_schema_uri (image classification): gs://google-cloud-aiplatform/schema/dataset/annotation/image_classification_1.0.0.yaml

Example split (filter split): {

“training_filter”: “labels.aiplatform.googleapis.com/ml_use=training”, “validation_filter”: “labels.aiplatform.googleapis.com/ml_use=validation”, “test_filter”: “labels.aiplatform.googleapis.com/ml_use=test”,

} Example split (fraction split): {

“training_fraction”: 0.7, “validation_fraction”: 0.2, “test_fraction”: 0.1,

}

  • Parameters

    • output_dir (str) – Required. The Google Cloud Storage location where the output is to be written to. In the given directory a new directory will be created with name: export-data-<dataset-display-name>-<timestamp-of-export-call> where timestamp is in YYYYMMDDHHMMSS format. All export output will be written into that directory. Inside that directory, annotations with the same schema will be grouped into sub directories which are named with the corresponding annotations’ schema title. Inside these sub directories, a schema.yaml will be created to describe the output format.

      If the uri doesn’t end with ‘/’, a ‘/’ will be automatically appended. The directory is created if it doesn’t exist.

    • annotation_filter (str) – Optional. An expression for filtering what part of the Dataset is to be exported. Only Annotations that match this filter will be exported. The filter syntax is the same as in [ListAnnotations][DatasetService.ListAnnotations].

    • saved_query_id (str) – Optional. The ID of a SavedQuery (annotation set) under this Dataset used for filtering Annotations for training.

      Only used for custom training data export use cases. Only applicable to Datasets that have SavedQueries.

      Only Annotations that are associated with this SavedQuery are used in respectively training. When used in conjunction with annotations_filter, the Annotations used for training are filtered by both saved_query_id and annotations_filter.

      Only one of saved_query_id and annotation_schema_uri should be specified as both of them represent the same thing: problem type.

    • annotation_schema_uri (str) – Optional. The Cloud Storage URI that points to a YAML file describing the annotation schema. The schema is defined as an OpenAPI 3.0.2 Schema Object. The schema files that can be used here are found in gs://google-cloud-aiplatform/schema/dataset/annotation/, note that the chosen schema must be consistent with metadata_schema_uri of this Dataset.

      Only used for custom training data export use cases. Only applicable if this Dataset that have DataItems and Annotations.

      Only Annotations that both match this schema and belong to DataItems not ignored by the split method are used in respectively training, validation or test role, depending on the role of the DataItem they are on.

      When used in conjunction with annotations_filter, the Annotations used for training are filtered by both annotations_filter and annotation_schema_uri.

    • split (Union[Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)], **Dict[[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str), [float](https://python.readthedocs.io/en/latest/library/functions.html#float)]*]) – The instructions how the export data should be split between the training, validation and test sets.

  • Returns

    Response message for DatasetService.ExportData in Dictionary format.

  • Return type

    export_data_response (Dict)

property gca_resource(: [proto.message.Message](https://proto-plus-python.readthedocs.io/en/latest/reference/message.html#proto.message.Message )

The underlying resource proto representation.

import_data()

Upload data to existing managed dataset.

  • Parameters

    • gcs_source (Union[str, **Sequence[str]]) – Required. Google Cloud Storage URI(-s) to the input file(s). May contain wildcards. For more information on wildcards, see https://cloud.google.com/storage/docs/gsutil/addlhelp/WildcardNames. .. rubric:: Examples

      str: “gs://bucket/file.csv” Sequence[str]: [“gs://bucket/file1.csv”, “gs://bucket/file2.csv”]

    • import_schema_uri (str) – Required. Points to a YAML file stored on Google Cloud Storage describing the import format. Validation will be done against the schema. The schema is defined as an OpenAPI 3.0.2 Schema Object.

    • data_item_labels (Dict) – Labels that will be applied to newly imported DataItems. If an identical DataItem as one being imported already exists in the Dataset, then these labels will be appended to these of the already existing one, and if labels with identical key is imported before, the old label value will be overwritten. If two DataItems are identical in the same import data operation, the labels will be combined and if key collision happens in this case, one of the values will be picked randomly. Two DataItems are considered identical if their content bytes are identical (e.g. image bytes or pdf bytes). These labels will be overridden by Annotation labels specified inside index file referenced by import_schema_uri, e.g. jsonl file. This arg is not for specifying the annotation name or the training target of your data, but for some global labels of the dataset. E.g., ‘data_item_labels={“aiplatform.googleapis.com/ml_use”:”training”}’ specifies that all the uploaded data are used for training.

    • sync (bool) – Whether to execute this method synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

    • import_request_timeout (float) – Optional. The timeout for the import request in seconds.

  • Returns

    Instantiated representation of the managed dataset resource.

  • Return type

    dataset (Dataset)

property labels(: Dict[str, str )

User-defined labels containing metadata about this resource.

Read more about labels at https://goo.gl/xmQnxf

classmethod list(filter: Optional[str] = None, order_by: Optional[str] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

List all instances of this Dataset resource.

Example Usage:

aiplatform.TabularDataset.list(

filter=’labels.my_key=”my_value”’,
order_by=’display_name’

)

  • Parameters

    • filter (str) – Optional. An expression for filtering the results of the request. For field names both snake_case and camelCase are supported.

    • order_by (str) – Optional. A comma-separated list of fields to order by, sorted in ascending order. Use “desc” after a field name for descending. Supported fields: display_name, create_time, update_time

    • project (str) – Optional. Project to retrieve list from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to retrieve list from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to retrieve list. Overrides credentials set in aiplatform.init.

  • Returns

    List[base.VertexAiResourceNoun] - A list of Dataset resource objects

property metadata_schema_uri(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

The metadata schema uri of this dataset resource.

property name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Name of this resource.

property resource_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Full qualified resource name.

to_dict()

Returns the resource proto as a dictionary.

update(*, display_name: Optional[str] = None, labels: Optional[Dict[str, str]] = None, description: Optional[str] = None, update_request_timeout: Optional[float] = None)

Update the dataset.

Updatable fields:


    * `display_name`


    * `description`


    * `labels`

  • Parameters

    • display_name (str) – Optional. The user-defined name of the Dataset. The name can be up to 128 characters long and can be consist of any UTF-8 characters.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. Labels with user-defined metadata to organize your Tensorboards. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. No more than 64 user labels can be associated with one Tensorboard (System labels are excluded). See https://goo.gl/xmQnxf for more information and examples of labels. System reserved label keys are prefixed with “aiplatform.googleapis.com/” and are immutable.

    • description (str) – Optional. The description of the Dataset.

    • update_request_timeout (float) – Optional. The timeout for the update request in seconds.

  • Returns

    Updated dataset.

  • Return type

    dataset (Dataset)

property update_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was last updated.

wait()

Helper method that blocks until all futures are complete.

class google.cloud.aiplatform.TemporalFusionTransformerForecastingTrainingJob(display_name: Optional[str] = None, optimization_objective: Optional[str] = None, column_specs: Optional[Dict[str, str]] = None, column_transformations: Optional[List[Dict[str, Dict[str, str]]]] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None, labels: Optional[Dict[str, str]] = None, training_encryption_spec_key_name: Optional[str] = None, model_encryption_spec_key_name: Optional[str] = None)

Bases: google.cloud.aiplatform.training_jobs._ForecastingTrainingJob

Class to train Temporal Fusion Transformer (TFT) forecasting models.

Constructs a Forecasting Training Job.

  • Parameters

    • display_name (str) – Optional. The user-defined name of this TrainingPipeline.

    • optimization_objective (str) – Optional. Objective function the model is to be optimized towards. The training process creates a Model that optimizes the value of the objective function over the validation set. The supported optimization objectives: “minimize-rmse” (default) - Minimize root-mean-squared error (RMSE). “minimize-mae” - Minimize mean-absolute error (MAE). “minimize-rmsle” - Minimize root-mean-squared log error (RMSLE). “minimize-rmspe” - Minimize root-mean-squared percentage error (RMSPE). “minimize-wape-mae” - Minimize the combination of weighted absolute percentage error (WAPE)

      and mean-absolute-error (MAE).

      ”minimize-quantile-loss” - Minimize the quantile loss at the defined quantiles.

      (Set this objective to build quantile forecasts.)

    • column_specs (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. Alternative to column_transformations where the keys of the dict are column names and their respective values are one of AutoMLTabularTrainingJob.column_data_types. When creating transformation for BigQuery Struct column, the column should be flattened using “.” as the delimiter. Only columns with no child should have a transformation. If an input column has no transformations on it, such a column is ignored by the training, except for the targetColumn, which should have no transformations defined on. Only one of column_transformations or column_specs should be passed.

    • column_transformations (List[Dict[str, **Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]]]*) – Optional. Transformations to apply to the input columns (i.e. columns other than the targetColumn). Each transformation may produce multiple result values from the column’s value, and all are used for training. When creating transformation for BigQuery Struct column, the column should be flattened using “.” as the delimiter. Only columns with no child should have a transformation. If an input column has no transformations on it, such a column is ignored by the training, except for the targetColumn, which should have no transformations defined on. Only one of column_transformations or column_specs should be passed. Consider using column_specs as column_transformations will be deprecated eventually.

    • project (str) – Optional. Project to run training in. Overrides project set in aiplatform.init.

    • location (str) – Optional. Location to run training in. Overrides location set in aiplatform.init.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to run call training service. Overrides credentials set in aiplatform.init.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The labels with user-defined metadata to organize TrainingPipelines. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information and examples of labels.

    • training_encryption_spec_key_name (Optional[str]) – Optional. The Cloud KMS resource identifier of the customer managed encryption key used to protect the training pipeline. Has the form: projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key. The key needs to be in the same region as where the compute resource is created. If set, this TrainingPipeline will be secured by this key. Note: Model trained by this TrainingPipeline is also secured by this key if model_to_upload is not set separately. Overrides encryption_spec_key_name set in aiplatform.init.

    • model_encryption_spec_key_name (Optional[str]) – Optional. The Cloud KMS resource identifier of the customer managed encryption key used to protect the model. Has the form: projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key. The key needs to be in the same region as where the compute resource is created. If set, the trained Model will be secured by this key. Overrides encryption_spec_key_name set in aiplatform.init.

  • Raises

    ValueError – If both column_transformations and column_specs were provided.

cancel()

Starts asynchronous cancellation on the TrainingJob. The server makes a best effort to cancel the job, but success is not guaranteed. On successful cancellation, the TrainingJob is not deleted; instead it becomes a job with state set to CANCELLED.

  • Raises

    RuntimeError – If this TrainingJob has not started running.

client_class()

alias of google.cloud.aiplatform.utils.PipelineClientWithOverride

property create_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was created.

delete(sync: bool = True)

Deletes this Vertex AI resource. WARNING: This deletion is permanent.

  • Parameters

    sync (bool) – Whether to execute this deletion synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

property display_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Display name of this resource.

done()

Method indicating whether a job has completed.

  • Returns

    True if the job has completed.

property encryption_spec(: Optional[[google.cloud.aiplatform_v1.types.encryption_spec.EncryptionSpec](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.EncryptionSpec)_ )

Customer-managed encryption key options for this Vertex AI resource.

If this is set, then all resources created by this Vertex AI resource will be encrypted with the provided encryption key.

property end_time(: Optional[datetime.datetime )

Time when the TrainingJob resource entered the PIPELINE_STATE_SUCCEEDED, PIPELINE_STATE_FAILED, PIPELINE_STATE_CANCELLED state.

property error(: Optional[google.rpc.status_pb2.Status )

Detailed error info for this TrainingJob resource. Only populated when the TrainingJob’s state is PIPELINE_STATE_FAILED or PIPELINE_STATE_CANCELLED.

property evaluated_data_items_bigquery_uri(: Optional[str )

BigQuery location of exported evaluated examples from the Training Job :returns:

BigQuery uri for the exported evaluated examples if the export

feature is enabled for training.

None: If the export feature was not enabled for training.

  • Return type

    str

property gca_resource(: [proto.message.Message](https://proto-plus-python.readthedocs.io/en/latest/reference/message.html#proto.message.Message )

The underlying resource proto representation.

classmethod get(resource_name: str, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Get Training Job for the given resource_name.

  • Parameters

    • resource_name (str) – Required. A fully-qualified resource name or ID.

    • project (str) – Optional project to retrieve training job from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional location to retrieve training job from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Custom credentials to use to upload this model. Overrides credentials set in aiplatform.init.

  • Raises

    ValueError – If the retrieved training job’s training task definition doesn’t match the custom training task definition.

  • Returns

    A Vertex AI Training Job

get_model(sync=True)

Vertex AI Model produced by this training, if one was produced.

  • Parameters

    sync (bool) – Whether to execute this method synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

  • Returns

    Vertex AI Model produced by this training

  • Return type

    model

  • Raises

    RuntimeError – If training failed or if a model was not produced by this training.

property has_failed(: [bool](https://python.readthedocs.io/en/latest/library/functions.html#bool )

Returns True if training has failed.

False otherwise.

property labels(: Dict[str, str )

User-defined labels containing metadata about this resource.

Read more about labels at https://goo.gl/xmQnxf

classmethod list(filter: Optional[str] = None, order_by: Optional[str] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

List all instances of this TrainingJob resource.

Example Usage:

aiplatform.CustomTrainingJob.list(

filter=’display_name=”experiment_a27”’,
order_by=’create_time desc’

)

  • Parameters

    • filter (str) – Optional. An expression for filtering the results of the request. For field names both snake_case and camelCase are supported.

    • order_by (str) – Optional. A comma-separated list of fields to order by, sorted in ascending order. Use “desc” after a field name for descending. Supported fields: display_name, create_time, update_time

    • project (str) – Optional. Project to retrieve list from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to retrieve list from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to retrieve list. Overrides credentials set in aiplatform.init.

  • Returns

    List[VertexAiResourceNoun] - A list of TrainingJob resource objects

property name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Name of this resource.

property resource_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Full qualified resource name.

run(dataset: google.cloud.aiplatform.datasets.time_series_dataset.TimeSeriesDataset, target_column: str, time_column: str, time_series_identifier_column: str, unavailable_at_forecast_columns: List[str], available_at_forecast_columns: List[str], forecast_horizon: int, data_granularity_unit: str, data_granularity_count: int, training_fraction_split: Optional[float] = None, validation_fraction_split: Optional[float] = None, test_fraction_split: Optional[float] = None, predefined_split_column_name: Optional[str] = None, timestamp_split_column_name: Optional[str] = None, weight_column: Optional[str] = None, time_series_attribute_columns: Optional[List[str]] = None, context_window: Optional[int] = None, export_evaluated_data_items: bool = False, export_evaluated_data_items_bigquery_destination_uri: Optional[str] = None, export_evaluated_data_items_override_destination: bool = False, quantiles: Optional[List[float]] = None, validation_options: Optional[str] = None, budget_milli_node_hours: int = 1000, model_display_name: Optional[str] = None, model_labels: Optional[Dict[str, str]] = None, model_id: Optional[str] = None, parent_model: Optional[str] = None, is_default_version: Optional[bool] = True, model_version_aliases: Optional[Sequence[str]] = None, model_version_description: Optional[str] = None, additional_experiments: Optional[List[str]] = None, hierarchy_group_columns: Optional[List[str]] = None, hierarchy_group_total_weight: Optional[float] = None, hierarchy_temporal_total_weight: Optional[float] = None, hierarchy_group_temporal_total_weight: Optional[float] = None, window_column: Optional[str] = None, window_stride_length: Optional[int] = None, window_max_count: Optional[int] = None, holiday_regions: Optional[List[str]] = None, sync: bool = True, create_request_timeout: Optional[float] = None, enable_probabilistic_inference: bool = False)

Runs the training job and returns a model.

If training on a Vertex AI dataset, you can use one of the following split configurations:

Data fraction splits:
Any of `training_fraction_split`, `validation_fraction_split` and
`test_fraction_split` may optionally be provided, they must sum to up to 1. If
the provided ones sum to less than 1, the remainder is assigned to sets as
decided by Vertex AI. If none of the fractions are set, by default roughly 80%
of data will be used for training, 10% for validation, and 10% for test.

Predefined splits:
Assigns input data to training, validation, and test sets based on the value of a provided key.
If using predefined splits, `predefined_split_column_name` must be provided.
Supported only for tabular Datasets.

Timestamp splits:
Assigns input data to training, validation, and test sets
based on a provided timestamps. The youngest data pieces are
assigned to training set, next to validation set, and the oldest
to the test set.
Supported only for tabular Datasets.

  • Parameters

    • dataset (datasets.TimeSeriesDataset) – Required. The dataset within the same Project from which data will be used to train the Model. The Dataset must use schema compatible with Model being trained, and what is compatible should be described in the used TrainingPipeline’s [training_task_definition] [google.cloud.aiplatform.v1beta1.TrainingPipeline.training_task_definition]. For time series Datasets, all their data is exported to training, to pick and choose from.

    • target_column (str) – Required. Name of the column that the Model is to predict values for. This column must be unavailable at forecast.

    • time_column (str) – Required. Name of the column that identifies time order in the time series. This column must be available at forecast.

    • time_series_identifier_column (str) – Required. Name of the column that identifies the time series.

    • unavailable_at_forecast_columns (List[str]) – Required. Column names of columns that are unavailable at forecast. Each column contains information for the given entity (identified by the [time_series_identifier_column]) that is unknown before the forecast (e.g. population of a city in a given year, or weather on a given day).

    • available_at_forecast_columns (List[str]) – Required. Column names of columns that are available at forecast. Each column contains information for the given entity (identified by the [time_series_identifier_column]) that is known at forecast.

    • forecast_horizon – (int): Required. The amount of time into the future for which forecasted values for the target are returned. Expressed in number of units defined by the [data_granularity_unit] and [data_granularity_count] field. Inclusive.

    • data_granularity_unit (str) – Required. The data granularity unit. Accepted values are minute, hour, day, week, month, year.

    • data_granularity_count (int) – Required. The number of data granularity units between data points in the training data. If [data_granularity_unit] is minute, can be 1, 5, 10, 15, or 30. For all other values of [data_granularity_unit], must be 1.

    • predefined_split_column_name (str) – Optional. The key is a name of one of the Dataset’s data columns. The value of the key (either the label’s value or value in the column) must be one of {TRAIN, VALIDATE, TEST}, and it defines to which set the given piece of data is assigned. If for a piece of data the key is not present or has an invalid value, that piece is ignored by the pipeline.

      Supported only for tabular and time series Datasets.

    • timestamp_split_column_name (str) – Optional. The key is a name of one of the Dataset’s data columns. The value of the key values of the key (the values in the column) must be in RFC 3339 date-time format, where time-offset = “Z” (e.g. 1985-04-12T23:20:50.52Z). If for a piece of data the key is not present or has an invalid value, that piece is ignored by the pipeline. Supported only for tabular and time series Datasets. This parameter must be used with training_fraction_split, validation_fraction_split, and test_fraction_split.

    • weight_column (str) – Optional. Name of the column that should be used as the weight column. Higher values in this column give more importance to the row during Model training. The column must have numeric values between 0 and 10000 inclusively, and 0 value means that the row is ignored. If the weight column field is not set, then all rows are assumed to have equal weight of 1.

    • time_series_attribute_columns (List[str]) – Optional. Column names that should be used as attribute columns. Each column is constant within a time series.

    • context_window (int) – Optional. The amount of time into the past training and prediction data is used for model training and prediction respectively. Expressed in number of units defined by the [data_granularity_unit] and [data_granularity_count] fields. When not provided uses the default value of 0 which means the model sets each series context window to be 0 (also known as “cold start”). Inclusive.

    • export_evaluated_data_items (bool) – Whether to export the test set predictions to a BigQuery table. If False, then the export is not performed.

    • export_evaluated_data_items_bigquery_destination_uri (string) – Optional. URI of desired destination BigQuery table for exported test set predictions.

      Expected format: bq://<project_id>:<dataset_id>:<table>

      If not specified, then results are exported to the following auto-created BigQuery table: <project_id>:export_evaluated_examples_<model_name>_<yyyy_MM_dd'T'HH_mm_ss_SSS'Z'>.evaluated_examples

      Applies only if [export_evaluated_data_items] is True.

    • export_evaluated_data_items_override_destination (bool) – Whether to override the contents of [export_evaluated_data_items_bigquery_destination_uri], if the table exists, for exported test set predictions. If False, and the table exists, then the training job will fail.

      Applies only if [export_evaluated_data_items] is True and [export_evaluated_data_items_bigquery_destination_uri] is specified.

    • quantiles (List[float]) – Quantiles to use for the minimize-quantile-loss [AutoMLForecastingTrainingJob.optimization_objective]. This argument is required in this case.

      Accepts up to 5 quantiles in the form of a double from 0 to 1, exclusive. Each quantile must be unique.

    • validation_options (str) – Validation options for the data validation component. The available options are: “fail-pipeline” - (default), will validate against the validation and fail the pipeline

      if it fails.

      ”ignore-validation” - ignore the results of the validation and continue the pipeline

    • budget_milli_node_hours (int) – Optional. The train budget of creating this Model, expressed in milli node hours i.e. 1,000 value in this field means 1 node hour. The training cost of the model will not exceed this budget. The final cost will be attempted to be close to the budget, though may end up being (even) noticeably smaller - at the backend’s discretion. This especially may happen when further model training ceases to provide any improvements. If the budget is set to a value known to be insufficient to train a Model for the given training set, the training won’t be attempted and will error. The minimum value is 1000 and the maximum is 72000.

    • model_display_name (str) – Optional. If the script produces a managed Vertex AI Model. The display name of the Model. The name can be up to 128 characters long and can be consist of any UTF-8 characters.

      If not provided upon creation, the job’s display_name is used.

    • model_labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The labels with user-defined metadata to organize your Models. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information and examples of labels.

    • model_id (str) – Optional. The ID to use for the Model produced by this job, which will become the final component of the model resource name. This value may be up to 63 characters, and valid characters are [a-z0-9_-]. The first character cannot be a number or hyphen.

    • parent_model (str) – Optional. The resource name or model ID of an existing model. The new model uploaded by this job will be a version of parent_model.

      Only set this field when training a new version of an existing model.

    • is_default_version (bool) – Optional. When set to True, the newly uploaded model version will automatically have alias “default” included. Subsequent uses of the model produced by this job without a version specified will use this “default” version.

      When set to False, the “default” alias will not be moved. Actions targeting the model version produced by this job will need to specifically reference this version by ID or alias.

      New model uploads, i.e. version 1, will always be “default” aliased.

    • model_version_aliases (Sequence[str]) – Optional. User provided version aliases so that the model version uploaded by this job can be referenced via alias instead of auto-generated version ID. A default version alias will be created for the first version of the model.

      The format is [a-z][a-zA-Z0-9-]{0,126}[a-z0-9]

    • model_version_description (str) – Optional. The description of the model version being uploaded by this job.

    • additional_experiments (List[str]) – Optional. Additional experiment flags for the time series forcasting training.

    • create_request_timeout (float) – Optional. The timeout for the create request in seconds.

    • hierarchy_group_columns (List[str]) – Optional. A list of time series attribute column names that define the time series hierarchy. Only one level of hierarchy is supported, ex. region for a hierarchy of stores or department for a hierarchy of products. If multiple columns are specified, time series will be grouped by their combined values, ex. (blue, large) for color and size, up to 5 columns are accepted. If no group columns are specified, all time series are considered to be part of the same group.

    • hierarchy_group_total_weight (float) – Optional. The weight of the loss for predictions aggregated over time series in the same hierarchy group.

    • hierarchy_temporal_total_weight (float) – Optional. The weight of the loss for predictions aggregated over the horizon for a single time series.

    • hierarchy_group_temporal_total_weight (float) – Optional. The weight of the loss for predictions aggregated over both the horizon and time series in the same hierarchy group.

    • window_column (str) – Optional. Name of the column that should be used to filter input rows. The column should contain either booleans or string booleans; if the value of the row is True, generate a sliding window from that row.

    • window_stride_length (int) – Optional. Step length used to generate input examples. Every window_stride_length rows will be used to generate a sliding window.

    • window_max_count (int) – Optional. Number of rows that should be used to generate input examples. If the total row count is larger than this number, the input data will be randomly sampled to hit the count.

    • holiday_regions (List[str]) – Optional. The geographical regions to use when creating holiday features. This option is only allowed when data_granularity_unit is day. Acceptable values can come from any of the following levels:

      Top level: GLOBAL Second level: continental regions

      NA: North America JAPAC: Japan and Asia Pacific EMEA: Europe, the Middle East and Africa LAC: Latin America and the Caribbean

      Third level: countries from ISO 3166-1 Country codes.

    • sync (bool) – Whether to execute this method synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

    • enable_probabilistic_inference (bool) – If probabilistic inference is enabled, the model will fit a distribution that captures the uncertainty of a prediction. At inference time, the predictive distribution is used to make a point prediction that minimizes the optimization objective. For example, the mean of a predictive distribution is the point prediction that minimizes RMSE loss. If quantiles are specified, then the quantiles of the distribution are also returned. The optimization objective cannot be minimize-quantile-loss.

  • Returns

    The trained Vertex AI Model resource or None if training did not

      produce a Vertex AI Model.

  • Return type

    model

  • Raises

    RuntimeError – If Training job has already been run or is waiting to run.

property start_time(: Optional[datetime.datetime )

Time when the TrainingJob entered the PIPELINE_STATE_RUNNING for the first time.

property state(: Optional[[google.cloud.aiplatform_v1.types.pipeline_state.PipelineState](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.PipelineState)_ )

Current training state.

to_dict()

Returns the resource proto as a dictionary.

property update_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was last updated.

wait()

Helper method that blocks until all futures are complete.

wait_for_resource_creation()

Waits until resource has been created.

class google.cloud.aiplatform.Tensorboard(tensorboard_name: str, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Bases: google.cloud.aiplatform.tensorboard.tensorboard_resource._TensorboardServiceResource

Managed tensorboard resource for Vertex AI.

Retrieves an existing managed tensorboard given a tensorboard name or ID.

  • Parameters

    • tensorboard_name (str) – Required. A fully-qualified tensorboard resource name or tensorboard ID. Example: “projects/123/locations/us-central1/tensorboards/456” or “456” when project and location are initialized or passed.

    • project (str) – Optional. Project to retrieve tensorboard from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to retrieve tensorboard from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to retrieve this Tensorboard. Overrides credentials set in aiplatform.init.

client_class()

alias of google.cloud.aiplatform.utils.TensorboardClientWithOverride

classmethod create(display_name: Optional[str] = None, description: Optional[str] = None, labels: Optional[Dict[str, str]] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None, request_metadata: Optional[Sequence[Tuple[str, str]]] = (), encryption_spec_key_name: Optional[str] = None, create_request_timeout: Optional[float] = None, is_default: bool = False)

Creates a new tensorboard.

Example Usage:

``

`

py tb = aiplatform.Tensorboard.create(

display_name=’my display name’, description=’my description’, labels={

‘key1’: ‘value1’, ‘key2’: ‘value2’

}

  • Parameters

    • display_name (str) – Optional. The user-defined name of the Tensorboard. The name can be up to 128 characters long and can be consist of any UTF-8 characters.

    • description (str) – Optional. Description of this Tensorboard.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. Labels with user-defined metadata to organize your Tensorboards. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. No more than 64 user labels can be associated with one Tensorboard (System labels are excluded). See https://goo.gl/xmQnxf for more information and examples of labels. System reserved label keys are prefixed with “aiplatform.googleapis.com/” and are immutable.

    • project (str) – Optional. Project to upload this model to. Overrides project set in aiplatform.init.

    • location (str) – Optional. Location to upload this model to. Overrides location set in aiplatform.init.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to upload this model. Overrides credentials set in aiplatform.init.

    • request_metadata (Sequence[Tuple[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*]) – Optional. Strings which should be sent along with the request as metadata.

    • encryption_spec_key_name (str) – Optional. Cloud KMS resource identifier of the customer managed encryption key used to protect the tensorboard. Has the form: projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key. The key needs to be in the same region as where the compute resource is created.

      If set, this Tensorboard and all sub-resources of this Tensorboard will be secured by this key.

      Overrides encryption_spec_key_name set in aiplatform.init.

    • create_request_timeout (float) – Optional. The timeout for the create request in seconds.

    • is_default (bool) – If the TensorBoard instance is default or not. The default TensorBoard instance will be used by Experiment/ExperimentRun when needed if no TensorBoard instance is explicitly specified.

  • Returns

    Instantiated representation of the managed tensorboard resource.

  • Return type

    tensorboard (Tensorboard)

property create_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was created.

delete(sync: bool = True)

Deletes this Vertex AI resource. WARNING: This deletion is permanent.

  • Parameters

    sync (bool) – Whether to execute this deletion synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

property display_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Display name of this resource.

property encryption_spec(: Optional[[google.cloud.aiplatform_v1.types.encryption_spec.EncryptionSpec](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.EncryptionSpec)_ )

Customer-managed encryption key options for this Vertex AI resource.

If this is set, then all resources created by this Vertex AI resource will be encrypted with the provided encryption key.

property gca_resource(: [proto.message.Message](https://proto-plus-python.readthedocs.io/en/latest/reference/message.html#proto.message.Message )

The underlying resource proto representation.

property labels(: Dict[str, str )

User-defined labels containing metadata about this resource.

Read more about labels at https://goo.gl/xmQnxf

classmethod list(filter: Optional[str] = None, order_by: Optional[str] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None, parent: Optional[str] = None)

List all instances of this Vertex AI Resource.

Example Usage:

aiplatform.BatchPredictionJobs.list(

filter=’state=”JOB_STATE_SUCCEEDED” AND display_name=”my_job”’,

)

aiplatform.Model.list(order_by=”create_time desc, display_name”)

  • Parameters

    • filter (str) – Optional. An expression for filtering the results of the request. For field names both snake_case and camelCase are supported.

    • order_by (str) – Optional. A comma-separated list of fields to order by, sorted in ascending order. Use “desc” after a field name for descending. Supported fields: display_name, create_time, update_time

    • project (str) – Optional. Project to retrieve list from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to retrieve list from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to retrieve list. Overrides credentials set in aiplatform.init.

    • parent (str) – Optional. The parent resource name if any to retrieve list from.

  • Returns

    List[VertexAiResourceNoun] - A list of SDK resource objects

property name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Name of this resource.

property resource_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Full qualified resource name.

to_dict()

Returns the resource proto as a dictionary.

update(display_name: Optional[str] = None, description: Optional[str] = None, labels: Optional[Dict[str, str]] = None, request_metadata: Optional[Sequence[Tuple[str, str]]] = (), encryption_spec_key_name: Optional[str] = None, is_default: Optional[bool] = None)

Updates an existing tensorboard.

Example Usage:

``

`

py tb = aiplatform.Tensorboard(tensorboard_name=’123456’) tb.update(

display_name=’update my display name’, description=’update my description’,

  • Parameters

    • display_name (str) – Optional. User-defined name of the Tensorboard. The name can be up to 128 characters long and can be consist of any UTF-8 characters.

    • description (str) – Optional. Description of this Tensorboard.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. Labels with user-defined metadata to organize your Tensorboards. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. No more than 64 user labels can be associated with one Tensorboard (System labels are excluded). See https://goo.gl/xmQnxf for more information and examples of labels. System reserved label keys are prefixed with “aiplatform.googleapis.com/” and are immutable.

    • request_metadata (Sequence[Tuple[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*]) – Optional. Strings which should be sent along with the request as metadata.

    • encryption_spec_key_name (str) – Optional. Cloud KMS resource identifier of the customer managed encryption key used to protect the tensorboard. Has the form: projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key. The key needs to be in the same region as where the compute resource is created.

      If set, this Tensorboard and all sub-resources of this Tensorboard will be secured by this key.

      Overrides encryption_spec_key_name set in aiplatform.init.

    • is_default (bool) – Optional. If the TensorBoard instance is default or not. The default TensorBoard instance will be used by Experiment/ExperimentRun when needed if no TensorBoard instance is explicitly specified.

  • Returns

    The managed tensorboard resource.

  • Return type

    Tensorboard

property update_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was last updated.

wait()

Helper method that blocks until all futures are complete.

class google.cloud.aiplatform.TensorboardExperiment(tensorboard_experiment_name: str, tensorboard_id: Optional[str] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Bases: google.cloud.aiplatform.tensorboard.tensorboard_resource._TensorboardServiceResource

Managed tensorboard resource for Vertex AI.

Retrieves an existing tensorboard experiment given a tensorboard experiment name or ID.

Example Usage:

``

`

py tb_exp = aiplatform.TensorboardExperiment(

tensorboard_experiment_name= “projects/123/locations/us-central1/tensorboards/456/experiments/678”

)

tb_exp = aiplatform.TensorboardExperiment(

tensorboard_experiment_name= “678”
tensorboard_id = “456”

  • Parameters

    • tensorboard_experiment_name (str) – Required. A fully-qualified tensorboard experiment resource name or resource ID. Example: “projects/123/locations/us-central1/tensorboards/456/experiments/678” or “678” when tensorboard_id is passed and project and location are initialized or passed.

    • tensorboard_id (str) – Optional. A tensorboard resource ID.

    • project (str) – Optional. Project to retrieve tensorboard from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to retrieve tensorboard from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to retrieve this Tensorboard. Overrides credentials set in aiplatform.init.

client_class()

alias of google.cloud.aiplatform.utils.TensorboardClientWithOverride

classmethod create(tensorboard_experiment_id: str, tensorboard_name: str, display_name: Optional[str] = None, description: Optional[str] = None, labels: Optional[Dict[str, str]] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None, request_metadata: Sequence[Tuple[str, str]] = (), create_request_timeout: Optional[float] = None)

Creates a new TensorboardExperiment.

Example Usage:

``

`

py tb_exp = aiplatform.TensorboardExperiment.create(

tensorboard_experiment_id=’my-experiment’ tensorboard_id=’456’ display_name=’my display name’, description=’my description’, labels={

‘key1’: ‘value1’, ‘key2’: ‘value2’

}

  • Parameters

    • tensorboard_experiment_id (str) – Required. The ID to use for the Tensorboard experiment, which will become the final component of the Tensorboard experiment’s resource name.

      This value should be 1-128 characters, and valid characters are /[a-z][0-9]-/.

      This corresponds to the tensorboard_experiment_id field on the request instance; if request is provided, this should not be set.

    • tensorboard_name (str) – Required. The resource name or ID of the Tensorboard to create the TensorboardExperiment in. Format of resource name: projects/{project}/locations/{location}/tensorboards/{tensorboard}

    • display_name (str) – Optional. The user-defined name of the Tensorboard Experiment. The name can be up to 128 characters long and can be consist of any UTF-8 characters.

    • description (str) – Optional. Description of this Tensorboard Experiment.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. Labels with user-defined metadata to organize your Tensorboards. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. No more than 64 user labels can be associated with one Tensorboard (System labels are excluded). See https://goo.gl/xmQnxf for more information and examples of labels. System reserved label keys are prefixed with “aiplatform.googleapis.com/” and are immutable.

    • project (str) – Optional. Project to upload this model to. Overrides project set in aiplatform.init.

    • location (str) – Optional. Location to upload this model to. Overrides location set in aiplatform.init.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to upload this model. Overrides credentials set in aiplatform.init.

    • request_metadata (Sequence[Tuple[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*]) – Optional. Strings which should be sent along with the request as metadata.

    • create_request_timeout (float) – Optional. The timeout for the create request in seconds.

  • Returns

    The TensorboardExperiment resource.

  • Return type

    TensorboardExperiment

property create_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was created.

delete(sync: bool = True)

Deletes this Vertex AI resource. WARNING: This deletion is permanent.

  • Parameters

    sync (bool) – Whether to execute this deletion synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

property display_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Display name of this resource.

property encryption_spec(: Optional[[google.cloud.aiplatform_v1.types.encryption_spec.EncryptionSpec](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.EncryptionSpec)_ )

Customer-managed encryption key options for this Vertex AI resource.

If this is set, then all resources created by this Vertex AI resource will be encrypted with the provided encryption key.

property gca_resource(: [proto.message.Message](https://proto-plus-python.readthedocs.io/en/latest/reference/message.html#proto.message.Message )

The underlying resource proto representation.

property labels(: Dict[str, str )

User-defined labels containing metadata about this resource.

Read more about labels at https://goo.gl/xmQnxf

classmethod list(tensorboard_name: str, filter: Optional[str] = None, order_by: Optional[str] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

List TensorboardExperiemnts in a Tensorboard resource.

``

`

py Example Usage:

aiplatform.TensorboardExperiment.list(

tensorboard_name=’projects/my-project/locations/us-central1/tensorboards/123’

)

``

`

  • Parameters

    • tensorboard_name (str) – Required. The resource name or resource ID of the Tensorboard to list TensorboardExperiments. Format, if resource name: ‘projects/{project}/locations/{location}/tensorboards/{tensorboard}’

    • filter (str) – Optional. An expression for filtering the results of the request. For field names both snake_case and camelCase are supported.

    • order_by (str) – Optional. A comma-separated list of fields to order by, sorted in ascending order. Use “desc” after a field name for descending. Supported fields: display_name, create_time, update_time

    • project (str) – Optional. Project to retrieve list from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to retrieve list from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to retrieve list. Overrides credentials set in aiplatform.init.

  • Returns

    List[TensorboardExperiment] - A list of TensorboardExperiments

property name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Name of this resource.

property resource_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Full qualified resource name.

to_dict()

Returns the resource proto as a dictionary.

property update_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was last updated.

wait()

Helper method that blocks until all futures are complete.

class google.cloud.aiplatform.TensorboardRun(tensorboard_run_name: str, tensorboard_id: Optional[str] = None, tensorboard_experiment_id: Optional[str] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Bases: google.cloud.aiplatform.tensorboard.tensorboard_resource._TensorboardServiceResource

Managed tensorboard resource for Vertex AI.

Retrieves an existing tensorboard run given a tensorboard run name or ID.

``

`

py Example Usage:

tb_run = aiplatform.TensorboardRun(

tensorboard_run_name= “projects/123/locations/us-central1/tensorboards/456/experiments/678/run/8910”

)

tb_run = aiplatform.TensorboardRun(

tensorboard_run_name= “8910”,
tensorboard_id = “456”,
tensorboard_experiment_id = “678”

)

``

`

  • Parameters

    • tensorboard_run_name (str) – Required. A fully-qualified tensorboard run resource name or resource ID. Example: “projects/123/locations/us-central1/tensorboards/456/experiments/678/runs/8910” or “8910” when tensorboard_id and tensorboard_experiment_id are passed and project and location are initialized or passed.

    • tensorboard_id (str) – Optional. A tensorboard resource ID.

    • tensorboard_experiment_id (str) – Optional. A tensorboard experiment resource ID.

    • project (str) – Optional. Project to retrieve tensorboard from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to retrieve tensorboard from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to retrieve this Tensorboard. Overrides credentials set in aiplatform.init.

  • Raises

    ValueError – if only one of tensorboard_id or tensorboard_experiment_id is provided.

client_class()

alias of google.cloud.aiplatform.utils.TensorboardClientWithOverride

classmethod create(tensorboard_run_id: str, tensorboard_experiment_name: str, tensorboard_id: Optional[str] = None, display_name: Optional[str] = None, description: Optional[str] = None, labels: Optional[Dict[str, str]] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None, request_metadata: Sequence[Tuple[str, str]] = (), create_request_timeout: Optional[float] = None)

Creates a new tensorboard run.

Example Usage:

``

`

py tb_run = aiplatform.TensorboardRun.create(

tensorboard_run_id=’my-run’ tensorboard_experiment_name=’my-experiment’ tensorboard_id=’456’ display_name=’my display name’, description=’my description’, labels={

‘key1’: ‘value1’, ‘key2’: ‘value2’

}

  • Parameters

    • tensorboard_run_id (str) – Required. The ID to use for the Tensorboard run, which will become the final component of the Tensorboard run’s resource name.

      This value should be 1-128 characters, and valid: characters are /[a-z][0-9]-/.

    • tensorboard_experiment_name (str) – Required. The resource name or ID of the TensorboardExperiment to create the TensorboardRun in. Resource name format: projects/{project}/locations/{location}/tensorboards/{tensorboard}/experiments/{experiment}

      If resource ID is provided then tensorboard_id must be provided.

    • tensorboard_id (str) – Optional. The resource ID of the Tensorboard to create the TensorboardRun in.

    • display_name (str) – Optional. The user-defined name of the Tensorboard Run. This value must be unique among all TensorboardRuns belonging to the same parent TensorboardExperiment.

      If not provided tensorboard_run_id will be used.

    • description (str) – Optional. Description of this Tensorboard Run.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. Labels with user-defined metadata to organize your Tensorboards. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. No more than 64 user labels can be associated with one Tensorboard (System labels are excluded). See https://goo.gl/xmQnxf for more information and examples of labels. System reserved label keys are prefixed with “aiplatform.googleapis.com/” and are immutable.

    • project (str) – Optional. Project to upload this model to. Overrides project set in aiplatform.init.

    • location (str) – Optional. Location to upload this model to. Overrides location set in aiplatform.init.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to upload this model. Overrides credentials set in aiplatform.init.

    • request_metadata (Sequence[Tuple[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*]) – Optional. Strings which should be sent along with the request as metadata.

    • create_request_timeout (float) – Optional. The timeout for the create request in seconds.

  • Returns

    The TensorboardRun resource.

  • Return type

    TensorboardRun

create_tensorboard_time_series(display_name: str, value_type: Union[google.cloud.aiplatform_v1.types.tensorboard_time_series.TensorboardTimeSeries.ValueType, str] = 'SCALAR', plugin_name: str = 'scalars', plugin_data: Optional[bytes] = None, description: Optional[str] = None)

Creates a new tensorboard time series.

Example Usage:

``

`

py tb_ts = tensorboard_run.create_tensorboard_time_series(

display_name=’my display name’, tensorboard_run_name=’my-run’ tensorboard_id=’456’ tensorboard_experiment_id=’my-experiment’ description=’my description’, labels={

‘key1’: ‘value1’, ‘key2’: ‘value2’

}

  • Parameters

    • display_name (str) – Optional. User provided name of this TensorboardTimeSeries. This value should be unique among all TensorboardTimeSeries resources belonging to the same TensorboardRun resource (parent resource).

    • value_type (Union[gca_tensorboard_time_series.TensorboardTimeSeries.ValueType, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. Type of TensorboardTimeSeries value. One of ‘SCALAR’, ‘TENSOR’, ‘BLOB_SEQUENCE’.

    • plugin_name (str) – Optional. Name of the plugin this time series pertain to. Such as Scalar, Tensor, Blob.

    • plugin_data (bytes) – Optional. Data of the current plugin, with the size limited to 65KB.

    • description (str) – Optional. Description of this TensorboardTimeseries.

  • Returns

    The TensorboardTimeSeries resource.

  • Return type

    TensorboardTimeSeries

property create_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was created.

delete(sync: bool = True)

Deletes this Vertex AI resource. WARNING: This deletion is permanent.

  • Parameters

    sync (bool) – Whether to execute this deletion synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

property display_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Display name of this resource.

property encryption_spec(: Optional[[google.cloud.aiplatform_v1.types.encryption_spec.EncryptionSpec](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.EncryptionSpec)_ )

Customer-managed encryption key options for this Vertex AI resource.

If this is set, then all resources created by this Vertex AI resource will be encrypted with the provided encryption key.

property gca_resource(: [proto.message.Message](https://proto-plus-python.readthedocs.io/en/latest/reference/message.html#proto.message.Message )

The underlying resource proto representation.

property labels(: Dict[str, str )

User-defined labels containing metadata about this resource.

Read more about labels at https://goo.gl/xmQnxf

classmethod list(tensorboard_experiment_name: str, tensorboard_id: Optional[str] = None, filter: Optional[str] = None, order_by: Optional[str] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

List all instances of TensorboardRun in TensorboardExperiment.

Example Usage:

``

`

py aiplatform.TensorboardRun.list(

tensorboard_experiment_name=’projects/my-project/locations/us-central1/tensorboards/123/experiments/456’

  • Parameters

    • tensorboard_experiment_name (str) – Required. The resource name or resource ID of the TensorboardExperiment to list TensorboardRun. Format, if resource name: ‘projects/{project}/locations/{location}/tensorboards/{tensorboard}/experiments/{experiment}’

      If resource ID is provided then tensorboard_id must be provided.

    • tensorboard_id (str) – Optional. The resource ID of the Tensorboard that contains the TensorboardExperiment to list TensorboardRun.

    • filter (str) – Optional. An expression for filtering the results of the request. For field names both snake_case and camelCase are supported.

    • order_by (str) – Optional. A comma-separated list of fields to order by, sorted in ascending order. Use “desc” after a field name for descending. Supported fields: display_name, create_time, update_time

    • project (str) – Optional. Project to retrieve list from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to retrieve list from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to retrieve list. Overrides credentials set in aiplatform.init.

  • Returns

    List[TensorboardRun] - A list of TensorboardRun

property name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Name of this resource.

read_time_series_data()

Read the time series data of this run.

``

`

py time_series_data = tensorboard_run.read_time_series_data()

print(time_series_data[‘loss’].values[-1].scalar.value)

``

`

  • Returns

    Dictionary of time series metric id to TimeSeriesData.

property resource_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Full qualified resource name.

to_dict()

Returns the resource proto as a dictionary.

property update_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was last updated.

wait()

Helper method that blocks until all futures are complete.

write_tensorboard_scalar_data(time_series_data: Dict[str, float], step: int, wall_time: Optional[google.protobuf.timestamp_pb2.Timestamp] = None)

Writes tensorboard scalar data to this run.

  • Parameters

    • time_series_data (Dict[str, *[float](https://python.readthedocs.io/en/latest/library/functions.html#float)]*) – Required. Dictionary of where keys are TensorboardTimeSeries display name and values are the scalar value..

    • step (int) – Required. Step index of this data point within the run.

    • wall_time (timestamp_pb2.Timestamp) – Optional. Wall clock timestamp when this data point is generated by the end user.

      If not provided, this will be generated based on the value from time.time()

class google.cloud.aiplatform.TensorboardTimeSeries(tensorboard_time_series_name: str, tensorboard_id: Optional[str] = None, tensorboard_experiment_id: Optional[str] = None, tensorboard_run_id: Optional[str] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Bases: google.cloud.aiplatform.tensorboard.tensorboard_resource._TensorboardServiceResource

Managed tensorboard resource for Vertex AI.

Retrieves an existing tensorboard time series given a tensorboard time series name or ID.

Example Usage:

``

`

py tb_ts = aiplatform.TensorboardTimeSeries(

tensorboard_time_series_name=”projects/123/locations/us-central1/tensorboards/456/experiments/789/run/1011/timeSeries/mse”

)

tb_ts = aiplatform.TensorboardTimeSeries(

tensorboard_time_series_name= “mse”,
tensorboard_id = “456”,
tensorboard_experiment_id = “789”
tensorboard_run_id = “1011”

  • Parameters

    • tensorboard_time_series_name (str) – Required. A fully-qualified tensorboard time series resource name or resource ID. Example: “projects/123/locations/us-central1/tensorboards/456/experiments/789/run/1011/timeSeries/mse” or “mse” when tensorboard_id, tensorboard_experiment_id, tensorboard_run_id are passed and project and location are initialized or passed.

    • tensorboard_id (str) – Optional. A tensorboard resource ID.

    • tensorboard_experiment_id (str) – Optional. A tensorboard experiment resource ID.

    • tensorboard_run_id (str) – Optional. A tensorboard run resource ID.

    • project (str) – Optional. Project to retrieve tensorboard from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to retrieve tensorboard from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to retrieve this Tensorboard. Overrides credentials set in aiplatform.init.

  • Raises

    ValueError – if only one of tensorboard_id or tensorboard_experiment_id is provided.

client_class()

alias of google.cloud.aiplatform.utils.TensorboardClientWithOverride

classmethod create(display_name: str, tensorboard_run_name: str, tensorboard_id: Optional[str] = None, tensorboard_experiment_id: Optional[str] = None, value_type: Union[google.cloud.aiplatform_v1.types.tensorboard_time_series.TensorboardTimeSeries.ValueType, str] = 'SCALAR', plugin_name: str = 'scalars', plugin_data: Optional[bytes] = None, description: Optional[str] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Creates a new tensorboard time series.

Example Usage:

``

`

py tb_ts = aiplatform.TensorboardTimeSeries.create(

display_name=’my display name’, tensorboard_run_name=’my-run’ tensorboard_id=’456’ tensorboard_experiment_id=’my-experiment’ description=’my description’, labels={

‘key1’: ‘value1’, ‘key2’: ‘value2’

}

  • Parameters

    • display_name (str) – Optional. User provided name of this TensorboardTimeSeries. This value should be unique among all TensorboardTimeSeries resources belonging to the same TensorboardRun resource (parent resource).

    • tensorboard_run_name (str) – Required. The resource name or ID of the TensorboardRun to create the TensorboardTimeseries in. Resource name format: projects/{project}/locations/{location}/tensorboards/{tensorboard}/experiments/{experiment}/runs/{run}

      If resource ID is provided then tensorboard_id and tensorboard_experiment_id must be provided.

    • tensorboard_id (str) – Optional. The resource ID of the Tensorboard to create the TensorboardTimeSeries in.

    • tensorboard_experiment_id (str) – Optional. The ID of the TensorboardExperiment to create the TensorboardTimeSeries in.

    • value_type (Union[gca_tensorboard_time_series.TensorboardTimeSeries.ValueType, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. Type of TensorboardTimeSeries value. One of ‘SCALAR’, ‘TENSOR’, ‘BLOB_SEQUENCE’.

    • plugin_name (str) – Optional. Name of the plugin this time series pertain to.

    • plugin_data (bytes) – Optional. Data of the current plugin, with the size limited to 65KB.

    • description (str) – Optional. Description of this TensorboardTimeseries.

    • project (str) – Optional. Project to upload this model to. Overrides project set in aiplatform.init.

    • location (str) – Optional. Location to upload this model to. Overrides location set in aiplatform.init.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to upload this model. Overrides credentials set in aiplatform.init.

  • Returns

    The TensorboardTimeSeries resource.

  • Return type

    TensorboardTimeSeries

property create_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was created.

delete(sync: bool = True)

Deletes this Vertex AI resource. WARNING: This deletion is permanent.

  • Parameters

    sync (bool) – Whether to execute this deletion synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

property display_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Display name of this resource.

property encryption_spec(: Optional[[google.cloud.aiplatform_v1.types.encryption_spec.EncryptionSpec](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.EncryptionSpec)_ )

Customer-managed encryption key options for this Vertex AI resource.

If this is set, then all resources created by this Vertex AI resource will be encrypted with the provided encryption key.

property gca_resource(: [proto.message.Message](https://proto-plus-python.readthedocs.io/en/latest/reference/message.html#proto.message.Message )

The underlying resource proto representation.

property labels(: Dict[str, str )

User-defined labels containing metadata about this resource.

Read more about labels at https://goo.gl/xmQnxf

classmethod list(tensorboard_run_name: str, tensorboard_id: Optional[str] = None, tensorboard_experiment_id: Optional[str] = None, filter: Optional[str] = None, order_by: Optional[str] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

List all instances of TensorboardTimeSeries in TensorboardRun.

Example Usage:

``

`

py aiplatform.TensorboardTimeSeries.list(

tensorboard_run_name=’projects/my-project/locations/us-central1/tensorboards/123/experiments/my-experiment/runs/my-run’

  • Parameters

    • tensorboard_run_name (str) – Required. The resource name or ID of the TensorboardRun to list the TensorboardTimeseries from. Resource name format: projects/{project}/locations/{location}/tensorboards/{tensorboard}/experiments/{experiment}/runs/{run}

      If resource ID is provided then tensorboard_id and tensorboard_experiment_id must be provided.

    • tensorboard_id (str) – Optional. The resource ID of the Tensorboard to list the TensorboardTimeSeries from.

    • tensorboard_experiment_id (str) – Optional. The ID of the TensorboardExperiment to list the TensorboardTimeSeries from.

    • filter (str) – Optional. An expression for filtering the results of the request. For field names both snake_case and camelCase are supported.

    • order_by (str) – Optional. A comma-separated list of fields to order by, sorted in ascending order. Use “desc” after a field name for descending. Supported fields: display_name, create_time, update_time

    • project (str) – Optional. Project to retrieve list from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to retrieve list from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to retrieve list. Overrides credentials set in aiplatform.init.

  • Returns

    List[TensorboardTimeSeries] - A list of TensorboardTimeSeries

property name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Name of this resource.

property resource_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Full qualified resource name.

to_dict()

Returns the resource proto as a dictionary.

property update_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was last updated.

wait()

Helper method that blocks until all futures are complete.

class google.cloud.aiplatform.TextDataset(dataset_name: str, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Bases: google.cloud.aiplatform.datasets.dataset._Dataset

Managed text dataset resource for Vertex AI.

Retrieves an existing managed dataset given a dataset name or ID.

  • Parameters

    • dataset_name (str) – Required. A fully-qualified dataset resource name or dataset ID. Example: “projects/123/locations/us-central1/datasets/456” or “456” when project and location are initialized or passed.

    • project (str) – Optional project to retrieve dataset from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional location to retrieve dataset from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Custom credentials to use to retrieve this Dataset. Overrides credentials set in aiplatform.init.

client_class()

alias of google.cloud.aiplatform.utils.DatasetClientWithOverride

classmethod create(display_name: Optional[str] = None, gcs_source: Optional[Union[str, Sequence[str]]] = None, import_schema_uri: Optional[str] = None, data_item_labels: Optional[Dict] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None, request_metadata: Optional[Sequence[Tuple[str, str]]] = (), labels: Optional[Dict[str, str]] = None, encryption_spec_key_name: Optional[str] = None, sync: bool = True, create_request_timeout: Optional[float] = None)

Creates a new text dataset and optionally imports data into dataset when source and import_schema_uri are passed.

Example Usage:

ds = aiplatform.TextDataset.create(

    > display_name=’my-dataset’,
    > gcs_source=’gs://my-bucket/dataset.csv’,
    > import_schema_uri=aiplatform.schema.dataset.ioformat.text.multi_label_classification

    )

  • Parameters

    • display_name (str) – Optional. The user-defined name of the Dataset. The name can be up to 128 characters long and can be consist of any UTF-8 characters.

    • gcs_source (Union[str, **Sequence[str]]) – Google Cloud Storage URI(-s) to the input file(s).

      Examples

      str: “gs://bucket/file.csv” Sequence[str]: [“gs://bucket/file1.csv”, “gs://bucket/file2.csv”]

    • import_schema_uri (str) – Points to a YAML file stored on Google Cloud Storage describing the import format. Validation will be done against the schema. The schema is defined as an OpenAPI 3.0.2 Schema Object.

    • data_item_labels (Dict) – Labels that will be applied to newly imported DataItems. If an identical DataItem as one being imported already exists in the Dataset, then these labels will be appended to these of the already existing one, and if labels with identical key is imported before, the old label value will be overwritten. If two DataItems are identical in the same import data operation, the labels will be combined and if key collision happens in this case, one of the values will be picked randomly. Two DataItems are considered identical if their content bytes are identical (e.g. image bytes or pdf bytes). These labels will be overridden by Annotation labels specified inside index file referenced by import_schema_uri, e.g. jsonl file.

    • project (str) – Project to upload this dataset to. Overrides project set in aiplatform.init.

    • location (str) – Location to upload this dataset to. Overrides location set in aiplatform.init.

    • credentials (auth_credentials.Credentials) – Custom credentials to use to upload this dataset. Overrides credentials set in aiplatform.init.

    • request_metadata (Sequence[Tuple[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*]) – Strings which should be sent along with the request as metadata.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. Labels with user-defined metadata to organize your Tensorboards. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. No more than 64 user labels can be associated with one Tensorboard (System labels are excluded). See https://goo.gl/xmQnxf for more information and examples of labels. System reserved label keys are prefixed with “aiplatform.googleapis.com/” and are immutable.

    • encryption_spec_key_name (Optional[str]) – Optional. The Cloud KMS resource identifier of the customer managed encryption key used to protect the dataset. Has the form: projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key. The key needs to be in the same region as where the compute resource is created.

      If set, this Dataset and all sub-resources of this Dataset will be secured by this key.

      Overrides encryption_spec_key_name set in aiplatform.init.

    • create_request_timeout (float) – Optional. The timeout for the create request in seconds.

    • sync (bool) – Whether to execute this method synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

  • Returns

    Instantiated representation of the managed text dataset resource.

  • Return type

    text_dataset (TextDataset)

property create_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was created.

delete(sync: bool = True)

Deletes this Vertex AI resource. WARNING: This deletion is permanent.

  • Parameters

    sync (bool) – Whether to execute this deletion synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

property display_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Display name of this resource.

property encryption_spec(: Optional[[google.cloud.aiplatform_v1.types.encryption_spec.EncryptionSpec](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.EncryptionSpec)_ )

Customer-managed encryption key options for this Vertex AI resource.

If this is set, then all resources created by this Vertex AI resource will be encrypted with the provided encryption key.

export_data(output_dir: str)

Exports data to output dir to GCS.

  • Parameters

    output_dir (str) – Required. The Google Cloud Storage location where the output is to be written to. In the given directory a new directory will be created with name: export-data-<dataset-display-name>-<timestamp-of-export-call> where timestamp is in YYYYMMDDHHMMSS format. All export output will be written into that directory. Inside that directory, annotations with the same schema will be grouped into sub directories which are named with the corresponding annotations’ schema title. Inside these sub directories, a schema.yaml will be created to describe the output format.

    If the uri doesn’t end with ‘/’, a ‘/’ will be automatically appended. The directory is created if it doesn’t exist.

  • Returns

    All of the files that are exported in this export operation.

  • Return type

    exported_files (Sequence[str])

export_data_for_custom_training(output_dir: str, annotation_filter: Optional[str] = None, saved_query_id: Optional[str] = None, annotation_schema_uri: Optional[str] = None, split: Optional[Union[Dict[str, str], Dict[str, float]]] = None)

Exports data to output dir to GCS for custom training use case.

Example annotation_schema_uri (image classification): gs://google-cloud-aiplatform/schema/dataset/annotation/image_classification_1.0.0.yaml

Example split (filter split): {

“training_filter”: “labels.aiplatform.googleapis.com/ml_use=training”, “validation_filter”: “labels.aiplatform.googleapis.com/ml_use=validation”, “test_filter”: “labels.aiplatform.googleapis.com/ml_use=test”,

} Example split (fraction split): {

“training_fraction”: 0.7, “validation_fraction”: 0.2, “test_fraction”: 0.1,

}

  • Parameters

    • output_dir (str) – Required. The Google Cloud Storage location where the output is to be written to. In the given directory a new directory will be created with name: export-data-<dataset-display-name>-<timestamp-of-export-call> where timestamp is in YYYYMMDDHHMMSS format. All export output will be written into that directory. Inside that directory, annotations with the same schema will be grouped into sub directories which are named with the corresponding annotations’ schema title. Inside these sub directories, a schema.yaml will be created to describe the output format.

      If the uri doesn’t end with ‘/’, a ‘/’ will be automatically appended. The directory is created if it doesn’t exist.

    • annotation_filter (str) – Optional. An expression for filtering what part of the Dataset is to be exported. Only Annotations that match this filter will be exported. The filter syntax is the same as in [ListAnnotations][DatasetService.ListAnnotations].

    • saved_query_id (str) – Optional. The ID of a SavedQuery (annotation set) under this Dataset used for filtering Annotations for training.

      Only used for custom training data export use cases. Only applicable to Datasets that have SavedQueries.

      Only Annotations that are associated with this SavedQuery are used in respectively training. When used in conjunction with annotations_filter, the Annotations used for training are filtered by both saved_query_id and annotations_filter.

      Only one of saved_query_id and annotation_schema_uri should be specified as both of them represent the same thing: problem type.

    • annotation_schema_uri (str) – Optional. The Cloud Storage URI that points to a YAML file describing the annotation schema. The schema is defined as an OpenAPI 3.0.2 Schema Object. The schema files that can be used here are found in gs://google-cloud-aiplatform/schema/dataset/annotation/, note that the chosen schema must be consistent with metadata_schema_uri of this Dataset.

      Only used for custom training data export use cases. Only applicable if this Dataset that have DataItems and Annotations.

      Only Annotations that both match this schema and belong to DataItems not ignored by the split method are used in respectively training, validation or test role, depending on the role of the DataItem they are on.

      When used in conjunction with annotations_filter, the Annotations used for training are filtered by both annotations_filter and annotation_schema_uri.

    • split (Union[Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)], **Dict[[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str), [float](https://python.readthedocs.io/en/latest/library/functions.html#float)]*]) – The instructions how the export data should be split between the training, validation and test sets.

  • Returns

    Response message for DatasetService.ExportData in Dictionary format.

  • Return type

    export_data_response (Dict)

property gca_resource(: [proto.message.Message](https://proto-plus-python.readthedocs.io/en/latest/reference/message.html#proto.message.Message )

The underlying resource proto representation.

import_data(gcs_source: Union[str, Sequence[str]], import_schema_uri: str, data_item_labels: Optional[Dict] = None, sync: bool = True, import_request_timeout: Optional[float] = None)

Upload data to existing managed dataset.

  • Parameters

    • gcs_source (Union[str, **Sequence[str]]) – Required. Google Cloud Storage URI(-s) to the input file(s). May contain wildcards. For more information on wildcards, see https://cloud.google.com/storage/docs/gsutil/addlhelp/WildcardNames. .. rubric:: Examples

      str: “gs://bucket/file.csv” Sequence[str]: [“gs://bucket/file1.csv”, “gs://bucket/file2.csv”]

    • import_schema_uri (str) – Required. Points to a YAML file stored on Google Cloud Storage describing the import format. Validation will be done against the schema. The schema is defined as an OpenAPI 3.0.2 Schema Object.

    • data_item_labels (Dict) – Labels that will be applied to newly imported DataItems. If an identical DataItem as one being imported already exists in the Dataset, then these labels will be appended to these of the already existing one, and if labels with identical key is imported before, the old label value will be overwritten. If two DataItems are identical in the same import data operation, the labels will be combined and if key collision happens in this case, one of the values will be picked randomly. Two DataItems are considered identical if their content bytes are identical (e.g. image bytes or pdf bytes). These labels will be overridden by Annotation labels specified inside index file referenced by import_schema_uri, e.g. jsonl file. This arg is not for specifying the annotation name or the training target of your data, but for some global labels of the dataset. E.g., ‘data_item_labels={“aiplatform.googleapis.com/ml_use”:”training”}’ specifies that all the uploaded data are used for training.

    • sync (bool) – Whether to execute this method synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

    • import_request_timeout (float) – Optional. The timeout for the import request in seconds.

  • Returns

    Instantiated representation of the managed dataset resource.

  • Return type

    dataset (Dataset)

property labels(: Dict[str, str )

User-defined labels containing metadata about this resource.

Read more about labels at https://goo.gl/xmQnxf

classmethod list(filter: Optional[str] = None, order_by: Optional[str] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

List all instances of this Dataset resource.

Example Usage:

aiplatform.TabularDataset.list(

filter=’labels.my_key=”my_value”’,
order_by=’display_name’

)

  • Parameters

    • filter (str) – Optional. An expression for filtering the results of the request. For field names both snake_case and camelCase are supported.

    • order_by (str) – Optional. A comma-separated list of fields to order by, sorted in ascending order. Use “desc” after a field name for descending. Supported fields: display_name, create_time, update_time

    • project (str) – Optional. Project to retrieve list from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to retrieve list from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to retrieve list. Overrides credentials set in aiplatform.init.

  • Returns

    List[base.VertexAiResourceNoun] - A list of Dataset resource objects

property metadata_schema_uri(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

The metadata schema uri of this dataset resource.

property name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Name of this resource.

property resource_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Full qualified resource name.

to_dict()

Returns the resource proto as a dictionary.

update(*, display_name: Optional[str] = None, labels: Optional[Dict[str, str]] = None, description: Optional[str] = None, update_request_timeout: Optional[float] = None)

Update the dataset.

Updatable fields:


    * `display_name`


    * `description`


    * `labels`

  • Parameters

    • display_name (str) – Optional. The user-defined name of the Dataset. The name can be up to 128 characters long and can be consist of any UTF-8 characters.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. Labels with user-defined metadata to organize your Tensorboards. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. No more than 64 user labels can be associated with one Tensorboard (System labels are excluded). See https://goo.gl/xmQnxf for more information and examples of labels. System reserved label keys are prefixed with “aiplatform.googleapis.com/” and are immutable.

    • description (str) – Optional. The description of the Dataset.

    • update_request_timeout (float) – Optional. The timeout for the update request in seconds.

  • Returns

    Updated dataset.

  • Return type

    dataset (Dataset)

property update_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was last updated.

wait()

Helper method that blocks until all futures are complete.

class google.cloud.aiplatform.TimeSeriesDataset(dataset_name: str, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Bases: google.cloud.aiplatform.datasets.column_names_dataset._ColumnNamesDataset

Managed time series dataset resource for Vertex AI

Retrieves an existing managed dataset given a dataset name or ID.

  • Parameters

    • dataset_name (str) – Required. A fully-qualified dataset resource name or dataset ID. Example: “projects/123/locations/us-central1/datasets/456” or “456” when project and location are initialized or passed.

    • project (str) – Optional project to retrieve dataset from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional location to retrieve dataset from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Custom credentials to use to retrieve this Dataset. Overrides credentials set in aiplatform.init.

client_class()

alias of google.cloud.aiplatform.utils.DatasetClientWithOverride

property column_names(: List[str )

Retrieve the columns for the dataset by extracting it from the Google Cloud Storage or Google BigQuery source.

  • Returns

    List[str]

      A list of columns names

  • Raises

    RuntimeError – When no valid source is found.

classmethod create(display_name: Optional[str] = None, gcs_source: Optional[Union[str, Sequence[str]]] = None, bq_source: Optional[str] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None, request_metadata: Optional[Sequence[Tuple[str, str]]] = (), labels: Optional[Dict[str, str]] = None, encryption_spec_key_name: Optional[str] = None, sync: bool = True, create_request_timeout: Optional[float] = None)

Creates a new time series dataset.

  • Parameters

    • display_name (str) – Optional. The user-defined name of the Dataset. The name can be up to 128 characters long and can be consist of any UTF-8 characters.

    • gcs_source (Union[str, **Sequence[str]]) – Google Cloud Storage URI(-s) to the input file(s).

      Examples

      str: “gs://bucket/file.csv” Sequence[str]: [“gs://bucket/file1.csv”, “gs://bucket/file2.csv”]

    • bq_source (str) – BigQuery URI to the input table. .. rubric:: Example

      ”bq://project.dataset.table_name”

    • project (str) – Project to upload this dataset to. Overrides project set in aiplatform.init.

    • location (str) – Location to upload this dataset to. Overrides location set in aiplatform.init.

    • credentials (auth_credentials.Credentials) – Custom credentials to use to upload this dataset. Overrides credentials set in aiplatform.init.

    • request_metadata (Sequence[Tuple[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*]) – Strings which should be sent along with the request as metadata.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. Labels with user-defined metadata to organize your datasets. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. No more than 64 user labels can be associated with one TimeSeriesDataset (System labels are excluded). See https://goo.gl/xmQnxf for more information and examples of labels. System reserved label keys are prefixed with “aiplatform.googleapis.com/” and are immutable.

    • encryption_spec_key_name (Optional[str]) – Optional. The Cloud KMS resource identifier of the customer managed encryption key used to protect the dataset. Has the form: projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key. The key needs to be in the same region as where the compute resource is created.

      If set, this Dataset and all sub-resources of this Dataset will be secured by this key.

      Overrides encryption_spec_key_name set in aiplatform.init.

    • sync (bool) – Whether to execute this method synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

    • create_request_timeout (float) – Optional. The timeout for the create request in seconds.

  • Returns

    Instantiated representation of the managed time series dataset resource.

  • Return type

    time_series_dataset (TimeSeriesDataset)

property create_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was created.

delete(sync: bool = True)

Deletes this Vertex AI resource. WARNING: This deletion is permanent.

  • Parameters

    sync (bool) – Whether to execute this deletion synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

property display_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Display name of this resource.

property encryption_spec(: Optional[[google.cloud.aiplatform_v1.types.encryption_spec.EncryptionSpec](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.EncryptionSpec)_ )

Customer-managed encryption key options for this Vertex AI resource.

If this is set, then all resources created by this Vertex AI resource will be encrypted with the provided encryption key.

export_data(output_dir: str)

Exports data to output dir to GCS.

  • Parameters

    output_dir (str) – Required. The Google Cloud Storage location where the output is to be written to. In the given directory a new directory will be created with name: export-data-<dataset-display-name>-<timestamp-of-export-call> where timestamp is in YYYYMMDDHHMMSS format. All export output will be written into that directory. Inside that directory, annotations with the same schema will be grouped into sub directories which are named with the corresponding annotations’ schema title. Inside these sub directories, a schema.yaml will be created to describe the output format.

    If the uri doesn’t end with ‘/’, a ‘/’ will be automatically appended. The directory is created if it doesn’t exist.

  • Returns

    All of the files that are exported in this export operation.

  • Return type

    exported_files (Sequence[str])

export_data_for_custom_training(output_dir: str, annotation_filter: Optional[str] = None, saved_query_id: Optional[str] = None, annotation_schema_uri: Optional[str] = None, split: Optional[Union[Dict[str, str], Dict[str, float]]] = None)

Exports data to output dir to GCS for custom training use case.

Example annotation_schema_uri (image classification): gs://google-cloud-aiplatform/schema/dataset/annotation/image_classification_1.0.0.yaml

Example split (filter split): {

“training_filter”: “labels.aiplatform.googleapis.com/ml_use=training”, “validation_filter”: “labels.aiplatform.googleapis.com/ml_use=validation”, “test_filter”: “labels.aiplatform.googleapis.com/ml_use=test”,

} Example split (fraction split): {

“training_fraction”: 0.7, “validation_fraction”: 0.2, “test_fraction”: 0.1,

}

  • Parameters

    • output_dir (str) – Required. The Google Cloud Storage location where the output is to be written to. In the given directory a new directory will be created with name: export-data-<dataset-display-name>-<timestamp-of-export-call> where timestamp is in YYYYMMDDHHMMSS format. All export output will be written into that directory. Inside that directory, annotations with the same schema will be grouped into sub directories which are named with the corresponding annotations’ schema title. Inside these sub directories, a schema.yaml will be created to describe the output format.

      If the uri doesn’t end with ‘/’, a ‘/’ will be automatically appended. The directory is created if it doesn’t exist.

    • annotation_filter (str) – Optional. An expression for filtering what part of the Dataset is to be exported. Only Annotations that match this filter will be exported. The filter syntax is the same as in [ListAnnotations][DatasetService.ListAnnotations].

    • saved_query_id (str) – Optional. The ID of a SavedQuery (annotation set) under this Dataset used for filtering Annotations for training.

      Only used for custom training data export use cases. Only applicable to Datasets that have SavedQueries.

      Only Annotations that are associated with this SavedQuery are used in respectively training. When used in conjunction with annotations_filter, the Annotations used for training are filtered by both saved_query_id and annotations_filter.

      Only one of saved_query_id and annotation_schema_uri should be specified as both of them represent the same thing: problem type.

    • annotation_schema_uri (str) – Optional. The Cloud Storage URI that points to a YAML file describing the annotation schema. The schema is defined as an OpenAPI 3.0.2 Schema Object. The schema files that can be used here are found in gs://google-cloud-aiplatform/schema/dataset/annotation/, note that the chosen schema must be consistent with metadata_schema_uri of this Dataset.

      Only used for custom training data export use cases. Only applicable if this Dataset that have DataItems and Annotations.

      Only Annotations that both match this schema and belong to DataItems not ignored by the split method are used in respectively training, validation or test role, depending on the role of the DataItem they are on.

      When used in conjunction with annotations_filter, the Annotations used for training are filtered by both annotations_filter and annotation_schema_uri.

    • split (Union[Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)], **Dict[[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str), [float](https://python.readthedocs.io/en/latest/library/functions.html#float)]*]) – The instructions how the export data should be split between the training, validation and test sets.

  • Returns

    Response message for DatasetService.ExportData in Dictionary format.

  • Return type

    export_data_response (Dict)

property gca_resource(: [proto.message.Message](https://proto-plus-python.readthedocs.io/en/latest/reference/message.html#proto.message.Message )

The underlying resource proto representation.

import_data()

Upload data to existing managed dataset.

  • Parameters

    • gcs_source (Union[str, **Sequence[str]]) – Required. Google Cloud Storage URI(-s) to the input file(s). May contain wildcards. For more information on wildcards, see https://cloud.google.com/storage/docs/gsutil/addlhelp/WildcardNames. .. rubric:: Examples

      str: “gs://bucket/file.csv” Sequence[str]: [“gs://bucket/file1.csv”, “gs://bucket/file2.csv”]

    • import_schema_uri (str) – Required. Points to a YAML file stored on Google Cloud Storage describing the import format. Validation will be done against the schema. The schema is defined as an OpenAPI 3.0.2 Schema Object.

    • data_item_labels (Dict) – Labels that will be applied to newly imported DataItems. If an identical DataItem as one being imported already exists in the Dataset, then these labels will be appended to these of the already existing one, and if labels with identical key is imported before, the old label value will be overwritten. If two DataItems are identical in the same import data operation, the labels will be combined and if key collision happens in this case, one of the values will be picked randomly. Two DataItems are considered identical if their content bytes are identical (e.g. image bytes or pdf bytes). These labels will be overridden by Annotation labels specified inside index file referenced by import_schema_uri, e.g. jsonl file. This arg is not for specifying the annotation name or the training target of your data, but for some global labels of the dataset. E.g., ‘data_item_labels={“aiplatform.googleapis.com/ml_use”:”training”}’ specifies that all the uploaded data are used for training.

    • sync (bool) – Whether to execute this method synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

    • import_request_timeout (float) – Optional. The timeout for the import request in seconds.

  • Returns

    Instantiated representation of the managed dataset resource.

  • Return type

    dataset (Dataset)

property labels(: Dict[str, str )

User-defined labels containing metadata about this resource.

Read more about labels at https://goo.gl/xmQnxf

classmethod list(filter: Optional[str] = None, order_by: Optional[str] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

List all instances of this Dataset resource.

Example Usage:

aiplatform.TabularDataset.list(

filter=’labels.my_key=”my_value”’,
order_by=’display_name’

)

  • Parameters

    • filter (str) – Optional. An expression for filtering the results of the request. For field names both snake_case and camelCase are supported.

    • order_by (str) – Optional. A comma-separated list of fields to order by, sorted in ascending order. Use “desc” after a field name for descending. Supported fields: display_name, create_time, update_time

    • project (str) – Optional. Project to retrieve list from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to retrieve list from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to retrieve list. Overrides credentials set in aiplatform.init.

  • Returns

    List[base.VertexAiResourceNoun] - A list of Dataset resource objects

property metadata_schema_uri(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

The metadata schema uri of this dataset resource.

property name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Name of this resource.

property resource_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Full qualified resource name.

to_dict()

Returns the resource proto as a dictionary.

update(*, display_name: Optional[str] = None, labels: Optional[Dict[str, str]] = None, description: Optional[str] = None, update_request_timeout: Optional[float] = None)

Update the dataset.

Updatable fields:


    * `display_name`


    * `description`


    * `labels`

  • Parameters

    • display_name (str) – Optional. The user-defined name of the Dataset. The name can be up to 128 characters long and can be consist of any UTF-8 characters.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. Labels with user-defined metadata to organize your Tensorboards. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. No more than 64 user labels can be associated with one Tensorboard (System labels are excluded). See https://goo.gl/xmQnxf for more information and examples of labels. System reserved label keys are prefixed with “aiplatform.googleapis.com/” and are immutable.

    • description (str) – Optional. The description of the Dataset.

    • update_request_timeout (float) – Optional. The timeout for the update request in seconds.

  • Returns

    Updated dataset.

  • Return type

    dataset (Dataset)

property update_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was last updated.

wait()

Helper method that blocks until all futures are complete.

class google.cloud.aiplatform.TimeSeriesDenseEncoderForecastingTrainingJob(display_name: Optional[str] = None, optimization_objective: Optional[str] = None, column_specs: Optional[Dict[str, str]] = None, column_transformations: Optional[List[Dict[str, Dict[str, str]]]] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None, labels: Optional[Dict[str, str]] = None, training_encryption_spec_key_name: Optional[str] = None, model_encryption_spec_key_name: Optional[str] = None)

Bases: google.cloud.aiplatform.training_jobs._ForecastingTrainingJob

Class to train Time series Dense Encoder (TiDE) forecasting models.

Constructs a Forecasting Training Job.

  • Parameters

    • display_name (str) – Optional. The user-defined name of this TrainingPipeline.

    • optimization_objective (str) – Optional. Objective function the model is to be optimized towards. The training process creates a Model that optimizes the value of the objective function over the validation set. The supported optimization objectives: “minimize-rmse” (default) - Minimize root-mean-squared error (RMSE). “minimize-mae” - Minimize mean-absolute error (MAE). “minimize-rmsle” - Minimize root-mean-squared log error (RMSLE). “minimize-rmspe” - Minimize root-mean-squared percentage error (RMSPE). “minimize-wape-mae” - Minimize the combination of weighted absolute percentage error (WAPE)

      and mean-absolute-error (MAE).

      ”minimize-quantile-loss” - Minimize the quantile loss at the defined quantiles.

      (Set this objective to build quantile forecasts.)

    • column_specs (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. Alternative to column_transformations where the keys of the dict are column names and their respective values are one of AutoMLTabularTrainingJob.column_data_types. When creating transformation for BigQuery Struct column, the column should be flattened using “.” as the delimiter. Only columns with no child should have a transformation. If an input column has no transformations on it, such a column is ignored by the training, except for the targetColumn, which should have no transformations defined on. Only one of column_transformations or column_specs should be passed.

    • column_transformations (List[Dict[str, **Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]]]*) – Optional. Transformations to apply to the input columns (i.e. columns other than the targetColumn). Each transformation may produce multiple result values from the column’s value, and all are used for training. When creating transformation for BigQuery Struct column, the column should be flattened using “.” as the delimiter. Only columns with no child should have a transformation. If an input column has no transformations on it, such a column is ignored by the training, except for the targetColumn, which should have no transformations defined on. Only one of column_transformations or column_specs should be passed. Consider using column_specs as column_transformations will be deprecated eventually.

    • project (str) – Optional. Project to run training in. Overrides project set in aiplatform.init.

    • location (str) – Optional. Location to run training in. Overrides location set in aiplatform.init.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to run call training service. Overrides credentials set in aiplatform.init.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The labels with user-defined metadata to organize TrainingPipelines. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information and examples of labels.

    • training_encryption_spec_key_name (Optional[str]) – Optional. The Cloud KMS resource identifier of the customer managed encryption key used to protect the training pipeline. Has the form: projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key. The key needs to be in the same region as where the compute resource is created. If set, this TrainingPipeline will be secured by this key. Note: Model trained by this TrainingPipeline is also secured by this key if model_to_upload is not set separately. Overrides encryption_spec_key_name set in aiplatform.init.

    • model_encryption_spec_key_name (Optional[str]) – Optional. The Cloud KMS resource identifier of the customer managed encryption key used to protect the model. Has the form: projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key. The key needs to be in the same region as where the compute resource is created. If set, the trained Model will be secured by this key. Overrides encryption_spec_key_name set in aiplatform.init.

  • Raises

    ValueError – If both column_transformations and column_specs were provided.

cancel()

Starts asynchronous cancellation on the TrainingJob. The server makes a best effort to cancel the job, but success is not guaranteed. On successful cancellation, the TrainingJob is not deleted; instead it becomes a job with state set to CANCELLED.

  • Raises

    RuntimeError – If this TrainingJob has not started running.

client_class()

alias of google.cloud.aiplatform.utils.PipelineClientWithOverride

property create_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was created.

delete(sync: bool = True)

Deletes this Vertex AI resource. WARNING: This deletion is permanent.

  • Parameters

    sync (bool) – Whether to execute this deletion synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

property display_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Display name of this resource.

done()

Method indicating whether a job has completed.

  • Returns

    True if the job has completed.

property encryption_spec(: Optional[[google.cloud.aiplatform_v1.types.encryption_spec.EncryptionSpec](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.EncryptionSpec)_ )

Customer-managed encryption key options for this Vertex AI resource.

If this is set, then all resources created by this Vertex AI resource will be encrypted with the provided encryption key.

property end_time(: Optional[datetime.datetime )

Time when the TrainingJob resource entered the PIPELINE_STATE_SUCCEEDED, PIPELINE_STATE_FAILED, PIPELINE_STATE_CANCELLED state.

property error(: Optional[google.rpc.status_pb2.Status )

Detailed error info for this TrainingJob resource. Only populated when the TrainingJob’s state is PIPELINE_STATE_FAILED or PIPELINE_STATE_CANCELLED.

property evaluated_data_items_bigquery_uri(: Optional[str )

BigQuery location of exported evaluated examples from the Training Job :returns:

BigQuery uri for the exported evaluated examples if the export

feature is enabled for training.

None: If the export feature was not enabled for training.

  • Return type

    str

property gca_resource(: [proto.message.Message](https://proto-plus-python.readthedocs.io/en/latest/reference/message.html#proto.message.Message )

The underlying resource proto representation.

classmethod get(resource_name: str, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Get Training Job for the given resource_name.

  • Parameters

    • resource_name (str) – Required. A fully-qualified resource name or ID.

    • project (str) – Optional project to retrieve training job from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional location to retrieve training job from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Custom credentials to use to upload this model. Overrides credentials set in aiplatform.init.

  • Raises

    ValueError – If the retrieved training job’s training task definition doesn’t match the custom training task definition.

  • Returns

    A Vertex AI Training Job

get_model(sync=True)

Vertex AI Model produced by this training, if one was produced.

  • Parameters

    sync (bool) – Whether to execute this method synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

  • Returns

    Vertex AI Model produced by this training

  • Return type

    model

  • Raises

    RuntimeError – If training failed or if a model was not produced by this training.

property has_failed(: [bool](https://python.readthedocs.io/en/latest/library/functions.html#bool )

Returns True if training has failed.

False otherwise.

property labels(: Dict[str, str )

User-defined labels containing metadata about this resource.

Read more about labels at https://goo.gl/xmQnxf

classmethod list(filter: Optional[str] = None, order_by: Optional[str] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

List all instances of this TrainingJob resource.

Example Usage:

aiplatform.CustomTrainingJob.list(

filter=’display_name=”experiment_a27”’,
order_by=’create_time desc’

)

  • Parameters

    • filter (str) – Optional. An expression for filtering the results of the request. For field names both snake_case and camelCase are supported.

    • order_by (str) – Optional. A comma-separated list of fields to order by, sorted in ascending order. Use “desc” after a field name for descending. Supported fields: display_name, create_time, update_time

    • project (str) – Optional. Project to retrieve list from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to retrieve list from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to retrieve list. Overrides credentials set in aiplatform.init.

  • Returns

    List[VertexAiResourceNoun] - A list of TrainingJob resource objects

property name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Name of this resource.

property resource_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Full qualified resource name.

run(dataset: google.cloud.aiplatform.datasets.time_series_dataset.TimeSeriesDataset, target_column: str, time_column: str, time_series_identifier_column: str, unavailable_at_forecast_columns: List[str], available_at_forecast_columns: List[str], forecast_horizon: int, data_granularity_unit: str, data_granularity_count: int, training_fraction_split: Optional[float] = None, validation_fraction_split: Optional[float] = None, test_fraction_split: Optional[float] = None, predefined_split_column_name: Optional[str] = None, timestamp_split_column_name: Optional[str] = None, weight_column: Optional[str] = None, time_series_attribute_columns: Optional[List[str]] = None, context_window: Optional[int] = None, export_evaluated_data_items: bool = False, export_evaluated_data_items_bigquery_destination_uri: Optional[str] = None, export_evaluated_data_items_override_destination: bool = False, quantiles: Optional[List[float]] = None, validation_options: Optional[str] = None, budget_milli_node_hours: int = 1000, model_display_name: Optional[str] = None, model_labels: Optional[Dict[str, str]] = None, model_id: Optional[str] = None, parent_model: Optional[str] = None, is_default_version: Optional[bool] = True, model_version_aliases: Optional[Sequence[str]] = None, model_version_description: Optional[str] = None, additional_experiments: Optional[List[str]] = None, hierarchy_group_columns: Optional[List[str]] = None, hierarchy_group_total_weight: Optional[float] = None, hierarchy_temporal_total_weight: Optional[float] = None, hierarchy_group_temporal_total_weight: Optional[float] = None, window_column: Optional[str] = None, window_stride_length: Optional[int] = None, window_max_count: Optional[int] = None, holiday_regions: Optional[List[str]] = None, sync: bool = True, create_request_timeout: Optional[float] = None, enable_probabilistic_inference: bool = False)

Runs the training job and returns a model.

If training on a Vertex AI dataset, you can use one of the following split configurations:

Data fraction splits:
Any of `training_fraction_split`, `validation_fraction_split` and
`test_fraction_split` may optionally be provided, they must sum to up to 1. If
the provided ones sum to less than 1, the remainder is assigned to sets as
decided by Vertex AI. If none of the fractions are set, by default roughly 80%
of data will be used for training, 10% for validation, and 10% for test.

Predefined splits:
Assigns input data to training, validation, and test sets based on the value of a provided key.
If using predefined splits, `predefined_split_column_name` must be provided.
Supported only for tabular Datasets.

Timestamp splits:
Assigns input data to training, validation, and test sets
based on a provided timestamps. The youngest data pieces are
assigned to training set, next to validation set, and the oldest
to the test set.
Supported only for tabular Datasets.

  • Parameters

    • dataset (datasets.TimeSeriesDataset) – Required. The dataset within the same Project from which data will be used to train the Model. The Dataset must use schema compatible with Model being trained, and what is compatible should be described in the used TrainingPipeline’s [training_task_definition] [google.cloud.aiplatform.v1beta1.TrainingPipeline.training_task_definition]. For time series Datasets, all their data is exported to training, to pick and choose from.

    • target_column (str) – Required. Name of the column that the Model is to predict values for. This column must be unavailable at forecast.

    • time_column (str) – Required. Name of the column that identifies time order in the time series. This column must be available at forecast.

    • time_series_identifier_column (str) – Required. Name of the column that identifies the time series.

    • unavailable_at_forecast_columns (List[str]) – Required. Column names of columns that are unavailable at forecast. Each column contains information for the given entity (identified by the [time_series_identifier_column]) that is unknown before the forecast (e.g. population of a city in a given year, or weather on a given day).

    • available_at_forecast_columns (List[str]) – Required. Column names of columns that are available at forecast. Each column contains information for the given entity (identified by the [time_series_identifier_column]) that is known at forecast.

    • forecast_horizon – (int): Required. The amount of time into the future for which forecasted values for the target are returned. Expressed in number of units defined by the [data_granularity_unit] and [data_granularity_count] field. Inclusive.

    • data_granularity_unit (str) – Required. The data granularity unit. Accepted values are minute, hour, day, week, month, year.

    • data_granularity_count (int) – Required. The number of data granularity units between data points in the training data. If [data_granularity_unit] is minute, can be 1, 5, 10, 15, or 30. For all other values of [data_granularity_unit], must be 1.

    • predefined_split_column_name (str) – Optional. The key is a name of one of the Dataset’s data columns. The value of the key (either the label’s value or value in the column) must be one of {TRAIN, VALIDATE, TEST}, and it defines to which set the given piece of data is assigned. If for a piece of data the key is not present or has an invalid value, that piece is ignored by the pipeline.

      Supported only for tabular and time series Datasets.

    • timestamp_split_column_name (str) – Optional. The key is a name of one of the Dataset’s data columns. The value of the key values of the key (the values in the column) must be in RFC 3339 date-time format, where time-offset = “Z” (e.g. 1985-04-12T23:20:50.52Z). If for a piece of data the key is not present or has an invalid value, that piece is ignored by the pipeline. Supported only for tabular and time series Datasets. This parameter must be used with training_fraction_split, validation_fraction_split, and test_fraction_split.

    • weight_column (str) – Optional. Name of the column that should be used as the weight column. Higher values in this column give more importance to the row during Model training. The column must have numeric values between 0 and 10000 inclusively, and 0 value means that the row is ignored. If the weight column field is not set, then all rows are assumed to have equal weight of 1.

    • time_series_attribute_columns (List[str]) – Optional. Column names that should be used as attribute columns. Each column is constant within a time series.

    • context_window (int) – Optional. The amount of time into the past training and prediction data is used for model training and prediction respectively. Expressed in number of units defined by the [data_granularity_unit] and [data_granularity_count] fields. When not provided uses the default value of 0 which means the model sets each series context window to be 0 (also known as “cold start”). Inclusive.

    • export_evaluated_data_items (bool) – Whether to export the test set predictions to a BigQuery table. If False, then the export is not performed.

    • export_evaluated_data_items_bigquery_destination_uri (string) – Optional. URI of desired destination BigQuery table for exported test set predictions.

      Expected format: bq://<project_id>:<dataset_id>:<table>

      If not specified, then results are exported to the following auto-created BigQuery table: <project_id>:export_evaluated_examples_<model_name>_<yyyy_MM_dd'T'HH_mm_ss_SSS'Z'>.evaluated_examples

      Applies only if [export_evaluated_data_items] is True.

    • export_evaluated_data_items_override_destination (bool) – Whether to override the contents of [export_evaluated_data_items_bigquery_destination_uri], if the table exists, for exported test set predictions. If False, and the table exists, then the training job will fail.

      Applies only if [export_evaluated_data_items] is True and [export_evaluated_data_items_bigquery_destination_uri] is specified.

    • quantiles (List[float]) – Quantiles to use for the minimize-quantile-loss [AutoMLForecastingTrainingJob.optimization_objective]. This argument is required in this case.

      Accepts up to 5 quantiles in the form of a double from 0 to 1, exclusive. Each quantile must be unique.

    • validation_options (str) – Validation options for the data validation component. The available options are: “fail-pipeline” - (default), will validate against the validation and fail the pipeline

      if it fails.

      ”ignore-validation” - ignore the results of the validation and continue the pipeline

    • budget_milli_node_hours (int) – Optional. The train budget of creating this Model, expressed in milli node hours i.e. 1,000 value in this field means 1 node hour. The training cost of the model will not exceed this budget. The final cost will be attempted to be close to the budget, though may end up being (even) noticeably smaller - at the backend’s discretion. This especially may happen when further model training ceases to provide any improvements. If the budget is set to a value known to be insufficient to train a Model for the given training set, the training won’t be attempted and will error. The minimum value is 1000 and the maximum is 72000.

    • model_display_name (str) – Optional. If the script produces a managed Vertex AI Model. The display name of the Model. The name can be up to 128 characters long and can be consist of any UTF-8 characters.

      If not provided upon creation, the job’s display_name is used.

    • model_labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. The labels with user-defined metadata to organize your Models. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information and examples of labels.

    • model_id (str) – Optional. The ID to use for the Model produced by this job, which will become the final component of the model resource name. This value may be up to 63 characters, and valid characters are [a-z0-9_-]. The first character cannot be a number or hyphen.

    • parent_model (str) – Optional. The resource name or model ID of an existing model. The new model uploaded by this job will be a version of parent_model.

      Only set this field when training a new version of an existing model.

    • is_default_version (bool) – Optional. When set to True, the newly uploaded model version will automatically have alias “default” included. Subsequent uses of the model produced by this job without a version specified will use this “default” version.

      When set to False, the “default” alias will not be moved. Actions targeting the model version produced by this job will need to specifically reference this version by ID or alias.

      New model uploads, i.e. version 1, will always be “default” aliased.

    • model_version_aliases (Sequence[str]) – Optional. User provided version aliases so that the model version uploaded by this job can be referenced via alias instead of auto-generated version ID. A default version alias will be created for the first version of the model.

      The format is [a-z][a-zA-Z0-9-]{0,126}[a-z0-9]

    • model_version_description (str) – Optional. The description of the model version being uploaded by this job.

    • additional_experiments (List[str]) – Optional. Additional experiment flags for the time series forcasting training.

    • create_request_timeout (float) – Optional. The timeout for the create request in seconds.

    • hierarchy_group_columns (List[str]) – Optional. A list of time series attribute column names that define the time series hierarchy. Only one level of hierarchy is supported, ex. region for a hierarchy of stores or department for a hierarchy of products. If multiple columns are specified, time series will be grouped by their combined values, ex. (blue, large) for color and size, up to 5 columns are accepted. If no group columns are specified, all time series are considered to be part of the same group.

    • hierarchy_group_total_weight (float) – Optional. The weight of the loss for predictions aggregated over time series in the same hierarchy group.

    • hierarchy_temporal_total_weight (float) – Optional. The weight of the loss for predictions aggregated over the horizon for a single time series.

    • hierarchy_group_temporal_total_weight (float) – Optional. The weight of the loss for predictions aggregated over both the horizon and time series in the same hierarchy group.

    • window_column (str) – Optional. Name of the column that should be used to filter input rows. The column should contain either booleans or string booleans; if the value of the row is True, generate a sliding window from that row.

    • window_stride_length (int) – Optional. Step length used to generate input examples. Every window_stride_length rows will be used to generate a sliding window.

    • window_max_count (int) – Optional. Number of rows that should be used to generate input examples. If the total row count is larger than this number, the input data will be randomly sampled to hit the count.

    • holiday_regions (List[str]) – Optional. The geographical regions to use when creating holiday features. This option is only allowed when data_granularity_unit is day. Acceptable values can come from any of the following levels:

      Top level: GLOBAL Second level: continental regions

      NA: North America JAPAC: Japan and Asia Pacific EMEA: Europe, the Middle East and Africa LAC: Latin America and the Caribbean

      Third level: countries from ISO 3166-1 Country codes.

    • sync (bool) – Whether to execute this method synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

    • enable_probabilistic_inference (bool) – If probabilistic inference is enabled, the model will fit a distribution that captures the uncertainty of a prediction. At inference time, the predictive distribution is used to make a point prediction that minimizes the optimization objective. For example, the mean of a predictive distribution is the point prediction that minimizes RMSE loss. If quantiles are specified, then the quantiles of the distribution are also returned. The optimization objective cannot be minimize-quantile-loss.

  • Returns

    The trained Vertex AI Model resource or None if training did not

      produce a Vertex AI Model.

  • Return type

    model

  • Raises

    RuntimeError – If Training job has already been run or is waiting to run.

property start_time(: Optional[datetime.datetime )

Time when the TrainingJob entered the PIPELINE_STATE_RUNNING for the first time.

property state(: Optional[[google.cloud.aiplatform_v1.types.pipeline_state.PipelineState](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.PipelineState)_ )

Current training state.

to_dict()

Returns the resource proto as a dictionary.

property update_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was last updated.

wait()

Helper method that blocks until all futures are complete.

wait_for_resource_creation()

Waits until resource has been created.

class google.cloud.aiplatform.VideoDataset(dataset_name: str, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Bases: google.cloud.aiplatform.datasets.dataset._Dataset

Managed video dataset resource for Vertex AI.

Retrieves an existing managed dataset given a dataset name or ID.

  • Parameters

    • dataset_name (str) – Required. A fully-qualified dataset resource name or dataset ID. Example: “projects/123/locations/us-central1/datasets/456” or “456” when project and location are initialized or passed.

    • project (str) – Optional project to retrieve dataset from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional location to retrieve dataset from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Custom credentials to use to retrieve this Dataset. Overrides credentials set in aiplatform.init.

client_class()

alias of google.cloud.aiplatform.utils.DatasetClientWithOverride

classmethod create(display_name: Optional[str] = None, gcs_source: Optional[Union[str, Sequence[str]]] = None, import_schema_uri: Optional[str] = None, data_item_labels: Optional[Dict] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None, request_metadata: Optional[Sequence[Tuple[str, str]]] = (), labels: Optional[Dict[str, str]] = None, encryption_spec_key_name: Optional[str] = None, sync: bool = True, create_request_timeout: Optional[float] = None)

Creates a new video dataset and optionally imports data into dataset when source and import_schema_uri are passed.

  • Parameters

    • display_name (str) – Optional. The user-defined name of the Dataset. The name can be up to 128 characters long and can be consist of any UTF-8 characters.

    • gcs_source (Union[str, **Sequence[str]]) – Google Cloud Storage URI(-s) to the input file(s).

      Examples

      str: “gs://bucket/file.csv” Sequence[str]: [“gs://bucket/file1.csv”, “gs://bucket/file2.csv”]

    • import_schema_uri (str) – Points to a YAML file stored on Google Cloud Storage describing the import format. Validation will be done against the schema. The schema is defined as an OpenAPI 3.0.2 Schema Object.

    • data_item_labels (Dict) – Labels that will be applied to newly imported DataItems. If an identical DataItem as one being imported already exists in the Dataset, then these labels will be appended to these of the already existing one, and if labels with identical key is imported before, the old label value will be overwritten. If two DataItems are identical in the same import data operation, the labels will be combined and if key collision happens in this case, one of the values will be picked randomly. Two DataItems are considered identical if their content bytes are identical (e.g. image bytes or pdf bytes). These labels will be overridden by Annotation labels specified inside index file referenced by import_schema_uri, e.g. jsonl file.

    • project (str) – Project to upload this dataset to. Overrides project set in aiplatform.init.

    • location (str) – Location to upload this dataset to. Overrides location set in aiplatform.init.

    • credentials (auth_credentials.Credentials) – Custom credentials to use to upload this dataset. Overrides credentials set in aiplatform.init.

    • request_metadata (Sequence[Tuple[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*]) – Strings which should be sent along with the request as metadata.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. Labels with user-defined metadata to organize your Tensorboards. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. No more than 64 user labels can be associated with one Tensorboard (System labels are excluded). See https://goo.gl/xmQnxf for more information and examples of labels. System reserved label keys are prefixed with “aiplatform.googleapis.com/” and are immutable.

    • encryption_spec_key_name (Optional[str]) – Optional. The Cloud KMS resource identifier of the customer managed encryption key used to protect the dataset. Has the form: projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key. The key needs to be in the same region as where the compute resource is created.

      If set, this Dataset and all sub-resources of this Dataset will be secured by this key.

      Overrides encryption_spec_key_name set in aiplatform.init.

    • create_request_timeout (float) – Optional. The timeout for the create request in seconds.

    • sync (bool) – Whether to execute this method synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

  • Returns

    Instantiated representation of the managed video dataset resource.

  • Return type

    video_dataset (VideoDataset)

property create_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was created.

delete(sync: bool = True)

Deletes this Vertex AI resource. WARNING: This deletion is permanent.

  • Parameters

    sync (bool) – Whether to execute this deletion synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

property display_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Display name of this resource.

property encryption_spec(: Optional[[google.cloud.aiplatform_v1.types.encryption_spec.EncryptionSpec](../aiplatform_v1/types.md#google.cloud.aiplatform_v1.types.EncryptionSpec)_ )

Customer-managed encryption key options for this Vertex AI resource.

If this is set, then all resources created by this Vertex AI resource will be encrypted with the provided encryption key.

export_data(output_dir: str)

Exports data to output dir to GCS.

  • Parameters

    output_dir (str) – Required. The Google Cloud Storage location where the output is to be written to. In the given directory a new directory will be created with name: export-data-<dataset-display-name>-<timestamp-of-export-call> where timestamp is in YYYYMMDDHHMMSS format. All export output will be written into that directory. Inside that directory, annotations with the same schema will be grouped into sub directories which are named with the corresponding annotations’ schema title. Inside these sub directories, a schema.yaml will be created to describe the output format.

    If the uri doesn’t end with ‘/’, a ‘/’ will be automatically appended. The directory is created if it doesn’t exist.

  • Returns

    All of the files that are exported in this export operation.

  • Return type

    exported_files (Sequence[str])

export_data_for_custom_training(output_dir: str, annotation_filter: Optional[str] = None, saved_query_id: Optional[str] = None, annotation_schema_uri: Optional[str] = None, split: Optional[Union[Dict[str, str], Dict[str, float]]] = None)

Exports data to output dir to GCS for custom training use case.

Example annotation_schema_uri (image classification): gs://google-cloud-aiplatform/schema/dataset/annotation/image_classification_1.0.0.yaml

Example split (filter split): {

“training_filter”: “labels.aiplatform.googleapis.com/ml_use=training”, “validation_filter”: “labels.aiplatform.googleapis.com/ml_use=validation”, “test_filter”: “labels.aiplatform.googleapis.com/ml_use=test”,

} Example split (fraction split): {

“training_fraction”: 0.7, “validation_fraction”: 0.2, “test_fraction”: 0.1,

}

  • Parameters

    • output_dir (str) – Required. The Google Cloud Storage location where the output is to be written to. In the given directory a new directory will be created with name: export-data-<dataset-display-name>-<timestamp-of-export-call> where timestamp is in YYYYMMDDHHMMSS format. All export output will be written into that directory. Inside that directory, annotations with the same schema will be grouped into sub directories which are named with the corresponding annotations’ schema title. Inside these sub directories, a schema.yaml will be created to describe the output format.

      If the uri doesn’t end with ‘/’, a ‘/’ will be automatically appended. The directory is created if it doesn’t exist.

    • annotation_filter (str) – Optional. An expression for filtering what part of the Dataset is to be exported. Only Annotations that match this filter will be exported. The filter syntax is the same as in [ListAnnotations][DatasetService.ListAnnotations].

    • saved_query_id (str) – Optional. The ID of a SavedQuery (annotation set) under this Dataset used for filtering Annotations for training.

      Only used for custom training data export use cases. Only applicable to Datasets that have SavedQueries.

      Only Annotations that are associated with this SavedQuery are used in respectively training. When used in conjunction with annotations_filter, the Annotations used for training are filtered by both saved_query_id and annotations_filter.

      Only one of saved_query_id and annotation_schema_uri should be specified as both of them represent the same thing: problem type.

    • annotation_schema_uri (str) – Optional. The Cloud Storage URI that points to a YAML file describing the annotation schema. The schema is defined as an OpenAPI 3.0.2 Schema Object. The schema files that can be used here are found in gs://google-cloud-aiplatform/schema/dataset/annotation/, note that the chosen schema must be consistent with metadata_schema_uri of this Dataset.

      Only used for custom training data export use cases. Only applicable if this Dataset that have DataItems and Annotations.

      Only Annotations that both match this schema and belong to DataItems not ignored by the split method are used in respectively training, validation or test role, depending on the role of the DataItem they are on.

      When used in conjunction with annotations_filter, the Annotations used for training are filtered by both annotations_filter and annotation_schema_uri.

    • split (Union[Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)], **Dict[[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str), [float](https://python.readthedocs.io/en/latest/library/functions.html#float)]*]) – The instructions how the export data should be split between the training, validation and test sets.

  • Returns

    Response message for DatasetService.ExportData in Dictionary format.

  • Return type

    export_data_response (Dict)

property gca_resource(: [proto.message.Message](https://proto-plus-python.readthedocs.io/en/latest/reference/message.html#proto.message.Message )

The underlying resource proto representation.

import_data(gcs_source: Union[str, Sequence[str]], import_schema_uri: str, data_item_labels: Optional[Dict] = None, sync: bool = True, import_request_timeout: Optional[float] = None)

Upload data to existing managed dataset.

  • Parameters

    • gcs_source (Union[str, **Sequence[str]]) – Required. Google Cloud Storage URI(-s) to the input file(s). May contain wildcards. For more information on wildcards, see https://cloud.google.com/storage/docs/gsutil/addlhelp/WildcardNames. .. rubric:: Examples

      str: “gs://bucket/file.csv” Sequence[str]: [“gs://bucket/file1.csv”, “gs://bucket/file2.csv”]

    • import_schema_uri (str) – Required. Points to a YAML file stored on Google Cloud Storage describing the import format. Validation will be done against the schema. The schema is defined as an OpenAPI 3.0.2 Schema Object.

    • data_item_labels (Dict) – Labels that will be applied to newly imported DataItems. If an identical DataItem as one being imported already exists in the Dataset, then these labels will be appended to these of the already existing one, and if labels with identical key is imported before, the old label value will be overwritten. If two DataItems are identical in the same import data operation, the labels will be combined and if key collision happens in this case, one of the values will be picked randomly. Two DataItems are considered identical if their content bytes are identical (e.g. image bytes or pdf bytes). These labels will be overridden by Annotation labels specified inside index file referenced by import_schema_uri, e.g. jsonl file. This arg is not for specifying the annotation name or the training target of your data, but for some global labels of the dataset. E.g., ‘data_item_labels={“aiplatform.googleapis.com/ml_use”:”training”}’ specifies that all the uploaded data are used for training.

    • sync (bool) – Whether to execute this method synchronously. If False, this method will be executed in concurrent Future and any downstream object will be immediately returned and synced when the Future has completed.

    • import_request_timeout (float) – Optional. The timeout for the import request in seconds.

  • Returns

    Instantiated representation of the managed dataset resource.

  • Return type

    dataset (Dataset)

property labels(: Dict[str, str )

User-defined labels containing metadata about this resource.

Read more about labels at https://goo.gl/xmQnxf

classmethod list(filter: Optional[str] = None, order_by: Optional[str] = None, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

List all instances of this Dataset resource.

Example Usage:

aiplatform.TabularDataset.list(

filter=’labels.my_key=”my_value”’,
order_by=’display_name’

)

  • Parameters

    • filter (str) – Optional. An expression for filtering the results of the request. For field names both snake_case and camelCase are supported.

    • order_by (str) – Optional. A comma-separated list of fields to order by, sorted in ascending order. Use “desc” after a field name for descending. Supported fields: display_name, create_time, update_time

    • project (str) – Optional. Project to retrieve list from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to retrieve list from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to retrieve list. Overrides credentials set in aiplatform.init.

  • Returns

    List[base.VertexAiResourceNoun] - A list of Dataset resource objects

property metadata_schema_uri(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

The metadata schema uri of this dataset resource.

property name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Name of this resource.

property resource_name(: [str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str )

Full qualified resource name.

to_dict()

Returns the resource proto as a dictionary.

update(*, display_name: Optional[str] = None, labels: Optional[Dict[str, str]] = None, description: Optional[str] = None, update_request_timeout: Optional[float] = None)

Update the dataset.

Updatable fields:


    * `display_name`


    * `description`


    * `labels`

  • Parameters

    • display_name (str) – Optional. The user-defined name of the Dataset. The name can be up to 128 characters long and can be consist of any UTF-8 characters.

    • labels (Dict[str, *[str](https://python.readthedocs.io/en/latest/library/stdtypes.html#str)]*) – Optional. Labels with user-defined metadata to organize your Tensorboards. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. No more than 64 user labels can be associated with one Tensorboard (System labels are excluded). See https://goo.gl/xmQnxf for more information and examples of labels. System reserved label keys are prefixed with “aiplatform.googleapis.com/” and are immutable.

    • description (str) – Optional. The description of the Dataset.

    • update_request_timeout (float) – Optional. The timeout for the update request in seconds.

  • Returns

    Updated dataset.

  • Return type

    dataset (Dataset)

property update_time(: [datetime.datetime](https://python.readthedocs.io/en/latest/library/datetime.html#datetime.datetime )

Time this resource was last updated.

wait()

Helper method that blocks until all futures are complete.

google.cloud.aiplatform.autolog(disable=False)

Enables autologging of parameters and metrics to Vertex Experiments.

After calling aiplatform.autolog(), any metrics and parameters from model training calls with supported ML frameworks will be automatically logged to Vertex Experiments.

Using autologging requires setting an experiment and experiment_tensorboard.

  • Parameters

    disable (bool) – Optional. Whether to disable autologging. Defaults to False. If set to True, this resets the MLFlow tracking URI to its previous state before autologging was called and remove logging filters.

  • Raises

    • ImportError – If MLFlow is not installed. MLFlow is required to use autologging in Vertex.

    • ValueError – If experiment or experiment_tensorboard is not set. If disable is passed and autologging hasn’t been enbaled.

google.cloud.aiplatform.end_run(state: google.cloud.aiplatform_v1.types.execution.Execution.State = State.COMPLETE)

Ends the the current experiment run.

\py aiplatform.start_run('my-run') ... aiplatform.end_run() ``

google.cloud.aiplatform.end_upload_tb_log()

Ends the current TensorBoard uploader

\py aiplatform.start_upload_tb_log(...) ... aiplatform.end_upload_tb_log() ``

google.cloud.aiplatform.get_experiment_df(experiment: Optional[str] = None)

Returns a Pandas DataFrame of the parameters and metrics associated with one experiment.

Example:

``

`

py aiplatform.init(experiment=’exp-1’) aiplatform.start_run(run=’run-1’) aiplatform.log_params({‘learning_rate’: 0.1}) aiplatform.log_metrics({‘accuracy’: 0.9})

aiplatform.start_run(run=’run-2’) aiplatform.log_params({‘learning_rate’: 0.2}) aiplatform.log_metrics({‘accuracy’: 0.95})

aiplatform.get_experiment_df()

``

`

Will result in the following DataFrame:

\ experiment_name | run_name | param.learning_rate | metric.accuracy exp-1 | run-1 | 0.1 | 0.9 exp-1 | run-2 | 0.2 | 0.95 ``

  • Parameters

    experiment (str) – Name of the Experiment to filter results. If not set, return results of current active experiment.

  • Returns

    Pandas Dataframe of Experiment with metrics and parameters.

  • Raises

    • NotFound exception if experiment does not exist. –

    • ValueError if given experiment is not associated with a wrong schema. –

google.cloud.aiplatform.get_experiment_model(artifact_id: str, *, metadata_store_id: str = 'default', project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Retrieves an existing ExperimentModel artifact given an artifact id.

  • Parameters

    • artifact_id (str) – Required. An artifact id of the ExperimentModel artifact.

    • metadata_store_id (str) – Optional. MetadataStore to retrieve Artifact from. If not set, metadata_store_id is set to “default”. If artifact_id is a fully-qualified resource name, its metadata_store_id overrides this one.

    • project (str) – Optional. Project to retrieve the artifact from. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location to retrieve the Artifact from. If not set, location set in aiplatform.init will be used.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials to use to retrieve this Artifact. Overrides credentials set in aiplatform.init.

  • Returns

    An ExperimentModel class that represents an Artifact resource.

  • Raises

    ValueError – if artifact’s schema title is not ‘google.ExperimentModel’.

google.cloud.aiplatform.get_pipeline_df(pipeline: str)

Returns a Pandas DataFrame of the parameters and metrics associated with one pipeline.

  • Parameters

    pipeline – Name of the Pipeline to filter results.

  • Returns

    Pandas Dataframe of Pipeline with metrics and parameters.

google.cloud.aiplatform.init(*, project: Optional[str] = None, location: Optional[str] = None, experiment: Optional[str] = None, experiment_description: Optional[str] = None, experiment_tensorboard: Optional[Union[str, google.cloud.aiplatform.tensorboard.tensorboard_resource.Tensorboard, bool]] = None, staging_bucket: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None, encryption_spec_key_name: Optional[str] = None, network: Optional[str] = None, service_account: Optional[str] = None, api_endpoint: Optional[str] = None, api_transport: Optional[str] = None)

Updates common initialization parameters with provided options.

  • Parameters

    • project (str) – The default project to use when making API calls.

    • location (str) – The default location to use when making API calls. If not set defaults to us-central-1.

    • experiment (str) – Optional. The experiment name.

    • experiment_description (str) – Optional. The description of the experiment.

    • experiment_tensorboard (Union[str, **tensorboard_resource.Tensorboard, *[bool](https://python.readthedocs.io/en/latest/library/functions.html#bool)]*) – Optional. The Vertex AI TensorBoard instance, Tensorboard resource name, or Tensorboard resource ID to use as a backing Tensorboard for the provided experiment.

      Example tensorboard resource name format: “projects/123/locations/us-central1/tensorboards/456”

      If experiment_tensorboard is provided and experiment is not, the provided experiment_tensorboard will be set as the global Tensorboard. Any subsequent calls to aiplatform.init() with experiment and without experiment_tensorboard will automatically assign the global Tensorboard to the experiment.

      If experiment_tensorboard is ommitted or set to True or None the global Tensorboard will be assigned to the experiment. If a global Tensorboard is not set, the default Tensorboard instance will be used, and created if it does not exist.

      To disable creating and using Tensorboard with experiment, set experiment_tensorboard to False. Any subsequent calls to aiplatform.init() should include this setting as well.

    • staging_bucket (str) – The default staging bucket to use to stage artifacts when making API calls. In the form gs://…

    • credentials (google.auth.credentials.Credentials) – The default custom credentials to use when making API calls. If not provided credentials will be ascertained from the environment.

    • encryption_spec_key_name (Optional[str]) – Optional. The Cloud KMS resource identifier of the customer managed encryption key used to protect a resource. Has the form: projects/my-project/locations/my-region/keyRings/my-kr/cryptoKeys/my-key. The key needs to be in the same region as where the compute resource is created.

      If set, this resource and all sub-resources will be secured by this key.

    • network (str) – Optional. The full name of the Compute Engine network to which jobs and resources should be peered. E.g. “projects/12345/global/networks/myVPC”. Private services access must already be configured for the network. If specified, all eligible jobs and resources created will be peered with this VPC.

    • service_account (str) – Optional. The service account used to launch jobs and deploy models. Jobs that use service_account: BatchPredictionJob, CustomJob, PipelineJob, HyperparameterTuningJob, CustomTrainingJob, CustomPythonPackageTrainingJob, CustomContainerTrainingJob, ModelEvaluationJob.

    • api_endpoint (str) – Optional. The desired API endpoint, e.g., us-central1-aiplatform.googleapis.com

    • api_transport (str) – Optional. The transport method which is either ‘grpc’ or ‘rest’. NOTE: “rest” transport functionality is currently in a beta state (preview).

  • Raises

    ValueError – If experiment_description is provided but experiment is not.

google.cloud.aiplatform.log(*, pipeline_job: Optional[google.cloud.aiplatform.pipeline_jobs.PipelineJob] = None)

Log Vertex AI Resources to the current experiment run.

\py aiplatform.start_run('my-run') my_job = aiplatform.PipelineJob(...) my_job.submit() aiplatform.log(my_job) ``

  • Parameters

    pipeline_job (pipeline_jobs.PipelineJob) – Optional. Vertex PipelineJob to associate to this Experiment Run.

google.cloud.aiplatform.log_classification_metrics(*, labels: Optional[List[str]] = None, matrix: Optional[List[List[int]]] = None, fpr: Optional[List[float]] = None, tpr: Optional[List[float]] = None, threshold: Optional[List[float]] = None, display_name: Optional[str] = None)

Create an artifact for classification metrics and log to ExperimentRun. Currently support confusion matrix and ROC curve.

``

`

py my_run = aiplatform.ExperimentRun(‘my-run’, experiment=’my-experiment’) classification_metrics = my_run.log_classification_metrics(

display_name=’my-classification-metrics’, labels=[‘cat’, ‘dog’], matrix=[[9, 1], [1, 9]], fpr=[0.1, 0.5, 0.9], tpr=[0.1, 0.7, 0.9], threshold=[0.9, 0.5, 0.1],

  • Parameters

    • labels (List[str]) – Optional. List of label names for the confusion matrix. Must be set if ‘matrix’ is set.

    • matrix (List[List[int]) – Optional. Values for the confusion matrix. Must be set if ‘labels’ is set.

    • fpr (List[float]) – Optional. List of false positive rates for the ROC curve. Must be set if ‘tpr’ or ‘thresholds’ is set.

    • tpr (List[float]) – Optional. List of true positive rates for the ROC curve. Must be set if ‘fpr’ or ‘thresholds’ is set.

    • threshold (List[float]) – Optional. List of thresholds for the ROC curve. Must be set if ‘fpr’ or ‘tpr’ is set.

    • display_name (str) – Optional. The user-defined name for the classification metric artifact.

  • Raises

    ValueError – if ‘labels’ and ‘matrix’ are not set together or if ‘labels’ and ‘matrix’ are not in the same length or if ‘fpr’ and ‘tpr’ and ‘threshold’ are not set together or if ‘fpr’ and ‘tpr’ and ‘threshold’ are not in the same length

google.cloud.aiplatform.log_metrics(metrics: Dict[str, Union[float, int, str]])

Log single or multiple Metrics with specified key and value pairs.

Metrics with the same key will be overwritten.

\py aiplatform.start_run('my-run', experiment='my-experiment') aiplatform.log_metrics({'accuracy': 0.9, 'recall': 0.8}) ``

google.cloud.aiplatform.log_model(model: Union[sklearn.base.BaseEstimator, xgb.Booster, tf.Module], artifact_id: Optional[str] = None, *, uri: Optional[str] = None, input_example: Union[list, dict, pd.DataFrame, np.ndarray] = None, display_name: Optional[str] = None, metadata_store_id: Optional[str] = 'default', project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Saves a ML model into a MLMD artifact and log it to this ExperimentRun.

Supported model frameworks: sklearn, xgboost, tensorflow.

Example usage:

``

`

py

model = LinearRegression() model.fit(X, y) aiplatform.init(

project=”my-project”, location=”my-location”, staging_bucket=”gs://my-bucket”, experiment=”my-exp”

) with aiplatform.start_run(“my-run”):

aiplatform.log_model(model, “my-sklearn-model”)

``

`

  • Parameters

    • model (Union["sklearn.base.BaseEstimator", **"xgb.Booster", **"tf.Module"]) – Required. A machine learning model.

    • artifact_id (str) – Optional. The resource id of the artifact. This id must be globally unique in a metadataStore. It may be up to 63 characters, and valid characters are [a-z0-9_-]. The first character cannot be a number or hyphen.

    • uri (str) – Optional. A gcs directory to save the model file. If not provided, gs://default-bucket/timestamp-uuid-frameworkName-model will be used. If default staging bucket is not set, a new bucket will be created.

    • input_example (Union[list, *[dict](https://python.readthedocs.io/en/latest/library/stdtypes.html#dict), pd.DataFrame, np.ndarray]*) – Optional. An example of a valid model input. Will be stored as a yaml file in the gcs uri. Accepts list, dict, pd.DataFrame, and np.ndarray The value inside a list must be a scalar or list. The value inside a dict must be a scalar, list, or np.ndarray.

    • display_name (str) – Optional. The display name of the artifact.

    • metadata_store_id (str) – Optional. The <metadata_store_id> portion of the resource name with the format: projects/123/locations/us-central1/metadataStores/<metadata_store_id>/artifacts/<resource_id> If not provided, the MetadataStore’s ID will be set to “default”.

    • project (str) – Optional. Project used to create this Artifact. Overrides project set in aiplatform.init.

    • location (str) – Optional. Location used to create this Artifact. Overrides location set in aiplatform.init.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials used to create this Artifact. Overrides credentials set in aiplatform.init.

  • Returns

    An ExperimentModel instance.

  • Raises

    ValueError – if model type is not supported.

google.cloud.aiplatform.log_params(params: Dict[str, Union[float, int, str]])

Log single or multiple parameters with specified key and value pairs.

Parameters with the same key will be overwritten.

\py aiplatform.start_run('my-run') aiplatform.log_params({'learning_rate': 0.1, 'dropout_rate': 0.2}) ``

google.cloud.aiplatform.log_time_series_metrics(metrics: Dict[str, float], step: Optional[int] = None, wall_time: Optional[google.protobuf.timestamp_pb2.Timestamp] = None)

Logs time series metrics to to this Experiment Run.

Requires the experiment or experiment run has a backing Vertex Tensorboard resource.

``

`

py my_tensorboard = aiplatform.Tensorboard(…) aiplatform.init(experiment=’my-experiment’, experiment_tensorboard=my_tensorboard) aiplatform.start_run(‘my-run’)

increments steps as logged

for i in range(10):

aiplatform.log_time_series_metrics({‘loss’: loss})

explicitly log steps

for i in range(10):

aiplatform.log_time_series_metrics({‘loss’: loss}, step=i)

``

`

  • Parameters

    • metrics (Dict[str, **Union[str, *[float](https://python.readthedocs.io/en/latest/library/functions.html#float)]*]) – Required. Dictionary of where keys are metric names and values are metric values.

    • step (int) – Optional. Step index of this data point within the run.

      If not provided, the latest step amongst all time series metrics already logged will be used.

    • wall_time (timestamp_pb2.Timestamp) – Optional. Wall clock timestamp when this data point is generated by the end user.

      If not provided, this will be generated based on the value from time.time()

  • Raises

    RuntimeError – If current experiment run doesn’t have a backing Tensorboard resource.

google.cloud.aiplatform.save_model(model: Union[sklearn.base.BaseEstimator, xgb.Booster, tf.Module], artifact_id: Optional[str] = None, *, uri: Optional[str] = None, input_example: Union[list, dict, pd.DataFrame, np.ndarray] = None, tf_save_model_kwargs: Optional[Dict[str, Any]] = None, display_name: Optional[str] = None, metadata_store_id: Optional[str] = 'default', project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Saves a ML model into a MLMD artifact.

Supported model frameworks: sklearn, xgboost, tensorflow.

Example usage:

aiplatform.init(project=”my-project”, location=”my-location”, staging_bucket=”gs://my-bucket”)
model = LinearRegression()
model.fit(X, y)
aiplatform.save_model(model, “my-sklearn-model”)

  • Parameters

    • model (Union["sklearn.base.BaseEstimator", **"xgb.Booster", **"tf.Module"]) – Required. A machine learning model.

    • artifact_id (str) – Optional. The resource id of the artifact. This id must be globally unique in a metadataStore. It may be up to 63 characters, and valid characters are [a-z0-9_-]. The first character cannot be a number or hyphen.

    • uri (str) – Optional. A gcs directory to save the model file. If not provided, gs://default-bucket/timestamp-uuid-frameworkName-model will be used. If default staging bucket is not set, a new bucket will be created.

    • input_example (Union[list, *[dict](https://python.readthedocs.io/en/latest/library/stdtypes.html#dict), pd.DataFrame, np.ndarray]*) – Optional. An example of a valid model input. Will be stored as a yaml file in the gcs uri. Accepts list, dict, pd.DataFrame, and np.ndarray The value inside a list must be a scalar or list. The value inside a dict must be a scalar, list, or np.ndarray.

    • tf_save_model_kwargs (Dict[str, **Any]) – Optional. A dict of kwargs to pass to the model’s save method. If saving a tf module, this will pass to “tf.saved_model.save” method. If saving a keras model, this will pass to “tf.keras.Model.save” method.

    • display_name (str) – Optional. The display name of the artifact.

    • metadata_store_id (str) – Optional. The <metadata_store_id> portion of the resource name with the format: projects/123/locations/us-central1/metadataStores/<metadata_store_id>/artifacts/<resource_id> If not provided, the MetadataStore’s ID will be set to “default”.

    • project (str) – Optional. Project used to create this Artifact. Overrides project set in aiplatform.init.

    • location (str) – Optional. Location used to create this Artifact. Overrides location set in aiplatform.init.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials used to create this Artifact. Overrides credentials set in aiplatform.init.

  • Returns

    An ExperimentModel instance.

  • Raises

    ValueError – if model type is not supported.

google.cloud.aiplatform.start_execution(*, schema_title: Optional[str] = None, display_name: Optional[str] = None, resource_id: Optional[str] = None, metadata: Optional[Dict[str, Any]] = None, schema_version: Optional[str] = None, description: Optional[str] = None, resume: bool = False, project: Optional[str] = None, location: Optional[str] = None, credentials: Optional[google.auth.credentials.Credentials] = None)

Create and starts a new Metadata Execution or resumes a previously created Execution.

To start a new execution:

``

`

py with aiplatform.start_execution(schema_title=’system.ContainerExecution’, display_name=’trainer) as exc:

exc.assign_input_artifacts([my_artifact]) model = aiplatform.Artifact.create(uri=’gs://my-uri’, schema_title=’system.Model’) exc.assign_output_artifacts([model])

``

`

To continue a previously created execution:

``

`

py with aiplatform.start_execution(resource_id=’my-exc’, resume=True) as exc:

…

``

` :param schema_title: Optional. schema_title identifies the schema title used by the Execution. Required if starting

a new Execution.

  • Parameters

    • resource_id (str) – Optional. The <resource_id> portion of the Execution name with the format. This is globally unique in a metadataStore: projects/123/locations/us-central1/metadataStores/<metadata_store_id>/executions/<resource_id>.

    • display_name (str) – Optional. The user-defined name of the Execution.

    • schema_version (str) – Optional. schema_version specifies the version used by the Execution. If not set, defaults to use the latest version.

    • metadata (Dict) – Optional. Contains the metadata information that will be stored in the Execution.

    • description (str) – Optional. Describes the purpose of the Execution to be created.

    • metadata_store_id (str) – Optional. The <metadata_store_id> portion of the resource name with the format: projects/123/locations/us-central1/metadataStores/<metadata_store_id>/artifacts/<resource_id> If not provided, the MetadataStore’s ID will be set to “default”.

    • project (str) – Optional. Project used to create this Execution. Overrides project set in aiplatform.init.

    • location (str) – Optional. Location used to create this Execution. Overrides location set in aiplatform.init.

    • credentials (auth_credentials.Credentials) – Optional. Custom credentials used to create this Execution. Overrides credentials set in aiplatform.init.

  • Returns

    Instantiated representation of the managed Metadata Execution.

  • Return type

    Execution

  • Raises

    • ValueError – If experiment run is set and project or location do not match experiment run.

    • ValueError – If resume set to True and resource_id is not provided.

    • ValueError – If creating a new executin and schema_title is not provided.

google.cloud.aiplatform.start_run(run: str, *, tensorboard: Optional[Union[google.cloud.aiplatform.tensorboard.tensorboard_resource.Tensorboard, str]] = None, resume=False)

Start a run to current session.

\py aiplatform.init(experiment='my-experiment') aiplatform.start_run('my-run') aiplatform.log_params({'learning_rate':0.1}) ``

Use as context manager. Run will be ended on context exit:

``

`

py aiplatform.init(experiment=’my-experiment’) with aiplatform.start_run(‘my-run’) as my_run:

my_run.log_params({‘learning_rate’:0.1})

``

`

Resume a previously started run:

``

`

py aiplatform.init(experiment=’my-experiment’) with aiplatform.start_run(‘my-run’, resume=True) as my_run:

my_run.log_params({‘learning_rate’:0.1})

``

`

  • Parameters

    • run (str) – Required. Name of the run to assign current session with.

    • Union[str (tensorboard) – Optional. Backing Tensorboard Resource to enable and store time series metrics logged to this Experiment Run using log_time_series_metrics.

      If not provided will the the default backing tensorboard of the currently set experiment.

    • tensorboard_resource.Tensorboard] – Optional. Backing Tensorboard Resource to enable and store time series metrics logged to this Experiment Run using log_time_series_metrics.

      If not provided will the the default backing tensorboard of the currently set experiment.

    • resume (bool) – Whether to resume this run. If False a new run will be created.

  • Raises

    ValueError – if experiment is not set. Or if run execution or metrics artifact is already created but with a different schema.

google.cloud.aiplatform.start_upload_tb_log(tensorboard_experiment_name: str, logdir: str, tensorboard_id: Optional[str] = None, project: Optional[str] = None, location: Optional[str] = None, experiment_display_name: Optional[str] = None, run_name_prefix: Optional[str] = None, description: Optional[str] = None, allowed_plugins: Optional[FrozenSet[str]] = None)

Continues to listen for new data in the logdir and uploads when it appears.

Note that after calling start_upload_tb_log() your thread will kept alive even if an exception is thrown. To ensure the thread gets shut down, put any code after start_upload_tb_log() and before end_upload_tb_log() in a try statement, and call end_upload_tb_log() in finally.

``

`

py Sample usage: aiplatform.init(location=’us-central1’, project=’my-project’) aiplatform.start_upload_tb_log(tensorboard_id=’123’,tensorboard_experiment_name=’my-experiment’,logdir=’my-logdir’)

try:

# your code here

finally:

aiplatform.end_upload_tb_log()

``

`

  • Parameters

    • tensorboard_experiment_name (str) – Required. Name of this tensorboard experiment. Unique to the given projects/{project}/locations/{location}/tensorboards/{tensorboard_id}.

    • logdir (str) – Required. path of the log directory to upload

    • tensorboard_id (str) – Optional. TensorBoard ID. If not set, tensorboard_id in aiplatform.init will be used.

    • project (str) – Optional. Project the TensorBoard is in. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location the TensorBoard is in. If not set, location set in aiplatform.init will be used.

    • experiment_display_name (str) – Optional. The display name of the experiment.

    • run_name_prefix (str) – Optional. If present, all runs created by this invocation will have their name prefixed by this value.

    • description (str) – Optional. String description to assign to the experiment.

    • allowed_plugins (FrozenSet[str]) – Optional. List of additional allowed plugin names.

google.cloud.aiplatform.upload_tb_log(tensorboard_experiment_name: str, logdir: str, tensorboard_id: Optional[str] = None, project: Optional[str] = None, location: Optional[str] = None, experiment_display_name: Optional[str] = None, run_name_prefix: Optional[str] = None, description: Optional[str] = None, verbosity: Optional[int] = 1, allowed_plugins: Optional[FrozenSet[str]] = None)

upload only the existing data in the logdir and then return immediately

\py Sample usage: aiplatform.init(location='us-central1', project='my-project') aiplatform.upload_tb_log(tensorboard_id='123',tensorboard_experiment_name='my-experiment',logdir='my-logdir') ``

  • Parameters

    • tensorboard_experiment_name (str) – Required. Name of this tensorboard experiment. Unique to the given projects/{project}/locations/{location}/tensorboards/{tensorboard_id}

    • logdir (str) – Required. The location of the TensorBoard logs that resides either in the local file system or Cloud Storage

    • tensorboard_id (str) – Optional. TensorBoard ID. If not set, tensorboard_id in aiplatform.init will be used.

    • project (str) – Optional. Project the TensorBoard is in. If not set, project set in aiplatform.init will be used.

    • location (str) – Optional. Location the TensorBoard is in. If not set, location set in aiplatform.init will be used.

    • experiment_display_name (str) – Optional. The display name of the experiment.

    • run_name_prefix (str) – Optional. If present, all runs created by this invocation will have their name prefixed by this value.

    • description (str) – Optional. String description to assign to the experiment.

    • verbosity (str) – Optional. Level of verbosity, an integer. Supported value: 0 - No upload statistics is printed. 1 - Print upload statistics

      while uploading data (default).

    • allowed_plugins (FrozenSet[str]) – Optional. List of additional allowed plugin names.