Class CustomJob (1.0.1)

CustomJob(
    display_name: str,
    worker_pool_specs: Union[
        List[Dict], List[google.cloud.aiplatform_v1.types.custom_job.WorkerPoolSpec]
    ],
    project: Optional[str] = None,
    location: Optional[str] = None,
    credentials: Optional[google.auth.credentials.Credentials] = None,
    encryption_spec_key_name: Optional[str] = None,
    staging_bucket: Optional[str] = None,
)

Vertex AI Custom Job.

Inheritance

builtins.object > google.cloud.aiplatform.base.VertexAiResourceNoun > builtins.object > google.cloud.aiplatform.base.FutureManager > google.cloud.aiplatform.base.VertexAiResourceNounWithFutureManager > google.cloud.aiplatform.jobs._Job > google.cloud.aiplatform.jobs._RunnableJob > CustomJob

Methods

CustomJob

CustomJob(
    display_name: str,
    worker_pool_specs: Union[
        List[Dict], List[google.cloud.aiplatform_v1.types.custom_job.WorkerPoolSpec]
    ],
    project: Optional[str] = None,
    location: Optional[str] = None,
    credentials: Optional[google.auth.credentials.Credentials] = None,
    encryption_spec_key_name: Optional[str] = None,
    staging_bucket: Optional[str] = None,
)

Cosntruct a Custom Job with Worker Pool Specs.

Example usage:
worker_pool_specs = [
        {
            "machine_spec": {
                "machine_type": "n1-standard-4",
                "accelerator_type": "NVIDIA_TESLA_K80",
                "accelerator_count": 1,
            },
            "replica_count": 1,
            "container_spec": {
                "image_uri": container_image_uri,
                "command": [],
                "args": [],
            },
        }
    ]

my_job = aiplatform.CustomJob(
    display_name='my_job',
    worker_pool_specs=worker_pool_specs
)

my_job.run()

For more information on configuring worker pool specs please visit: https://cloud.google.com/ai-platform-unified/docs/training/create-custom-job

Parameters

Name	Description
display_name	`str` Required. The user-defined name of the HyperparameterTuningJob. The name can be up to 128 characters long and can be consist of any UTF-8 characters.
worker_pool_specs	`Union[List[Dict], List[aiplatform.gapic.WorkerPoolSpec]]` Required. The spec of the worker pools including machine type and Docker image. Can provided as a list of dictionaries or list of WorkerPoolSpec proto messages.
project	`str` Optional.Project to run the custom job in. Overrides project set in aiplatform.init.
location	`str` Optional.Location to run the custom job in. Overrides location set in aiplatform.init.
credentials	`auth_credentials.Credentials` Optional.Custom credentials to use to run call custom job service. Overrides credentials set in aiplatform.init.
encryption_spec_key_name	`str` Optional.Customer-managed encryption key name for a CustomJob. If this is set, then all resources created by the CustomJob will be encrypted with the provided encryption key.
staging_bucket	`str` Optional. Bucket for produced custom job artifacts. Overrides staging_bucket set in aiplatform.init.

Exceptions

Type	Description
RuntimeErro	is not staging bucket was set using aiplatfrom.init and a staging:
bucke	was not passed in.:

from_local_script

from_local_script(
    display_name: str,
    script_path: str,
    container_uri: str,
    args: Optional[List[Union[float, int, str]]] = None,
    requirements: Optional[Sequence[str]] = None,
    environment_variables: Optional[Dict[str, str]] = None,
    replica_count: int = 1,
    machine_type: str = "n1-standard-4",
    accelerator_type: str = "ACCELERATOR_TYPE_UNSPECIFIED",
    accelerator_count: int = 0,
    project: Optional[str] = None,
    location: Optional[str] = None,
    credentials: Optional[google.auth.credentials.Credentials] = None,
    encryption_spec_key_name: Optional[str] = None,
    staging_bucket: Optional[str] = None,
)

Configures a custom job from a local script.

Example usage:

job = aiplatform.CustomJob.from_local_script(
    display_name="my-custom-job",
    script_path="training_script.py",
    container_uri="gcr.io/cloud-aiplatform/training/tf-cpu.2-2:latest",
    requirements=["gcsfs==0.7.1"],
    replica_count=1,
    args=['--dataset', 'gs://my-bucket/my-dataset',
    '--model_output_uri', 'gs://my-bucket/model']
)

job.run()

Parameters

Name	Description
display_name	`str` Required. The user-defined name of this CustomJob.
script_path	`str` Required. Local path to training script.
container_uri	`str` Required: Uri of the training container image to use for custom job.
args	`Optional[List[Union[str, float, int]]]` Optional. Command line arguments to be passed to the Python task.
requirements	`Sequence[str]` Optional. List of python packages dependencies of script.
environment_variables	`Dict[str, str]` Optional. Environment variables to be passed to the container. Should be a dictionary where keys are environment variable names and values are environment variable values for those names. At most 10 environment variables can be specified. The Name of the environment variable must be unique. environment_variables = { 'MY_KEY': 'MY_VALUE' }
replica_count	`int` Optional. The number of worker replicas. If replica count = 1 then one chief replica will be provisioned. If replica_count > 1 the remainder will be provisioned as a worker replica pool.
machine_type	`str` Optional. The type of machine to use for training.
accelerator_type	`str` Optional. Hardware accelerator type. One of ACCELERATOR_TYPE_UNSPECIFIED, NVIDIA_TESLA_K80, NVIDIA_TESLA_P100, NVIDIA_TESLA_V100, NVIDIA_TESLA_P4, NVIDIA_TESLA_T4
accelerator_count	`int` Optional. The number of accelerators to attach to a worker replica.
project	`str` Optional. Project to run the custom job in. Overrides project set in aiplatform.init.
location	`str` Optional. Location to run the custom job in. Overrides location set in aiplatform.init.
credentials	`auth_credentials.Credentials` Optional. Custom credentials to use to run call custom job service. Overrides credentials set in aiplatform.init.
encryption_spec_key_name	`str` Optional. Customer-managed encryption key name for a CustomJob. If this is set, then all resources created by the CustomJob will be encrypted with the provided encryption key.
staging_bucket	`str` Optional. Bucket for produced custom job artifacts. Overrides staging_bucket set in aiplatform.init.

Exceptions

Type	Description
RuntimeErro	is not staging bucket was set using aiplatfrom.init and a staging:
bucke	was not passed in.:

run

run(
    service_account: Optional[str] = None,
    network: Optional[str] = None,
    timeout: Optional[int] = None,
    restart_job_on_worker_restart: bool = False,
    tensorboard: Optional[str] = None,
    sync: bool = True,
)

Run this configured CustomJob.

Parameters

Name	Description
service_account	`str` Optional. Specifies the service account for workload run-as account. Users submitting jobs must have act-as permission on this run-as account.
network	`str` Optional. The full name of the Compute Engine network to which the job should be peered. For example, projects/12345/global/networks/myVPC. Private services access must already be configured for the network. If left unspecified, the job is not peered with any network.
timeout	`int` The maximum job running time in seconds. The default is 7 days.
restart_job_on_worker_restart	`bool` Restarts the entire CustomJob if a worker gets restarted. This feature can be used by distributed training jobs that are not resilient to workers leaving and joining a job.
tensorboard	`str` Optional. The name of an Vertex AI Tensorboard resource to which this CustomJob will upload Tensorboard logs. Format: `projects/{project}/locations/{location}/tensorboards/{tensorboard}` The training script should write Tensorboard to following Vertex AI environment variable: AIP_TENSORBOARD_LOG_DIR `service_account` is required with provided `tensorboard`. For more information on configuring your service account please visit: https://cloud.google.com/vertex-ai/docs/experiments/tensorboard-training
sync	`bool` Whether to execute this method synchronously. If False, this method will unblock and it will be executed in a concurrent Future.