Solicita recursos de máquina de Google Cloud con Vertex AI Pipelines
Organiza tus páginas con colecciones
Guarda y categoriza el contenido según tus preferencias.
Puedes ejecutar tu componente de Python en las Vertex AI Pipelines mediante los recursos de máquinas específicos de Google Cloud que ofrece el entrenamiento personalizado de Vertex AI.
Crea un trabajo de entrenamiento personalizado a partir de un componente mediante Vertex AI Pipelines
En el siguiente ejemplo, se muestra cómo usar el método create_custom_training_job_from_component para transformar un componente de Python en un trabajo de entrenamiento personalizado con recursos de máquina de Google Cloud definidos por el usuario y, luego, ejecutar la canalización compilada en Vertex AI Pipelines:
import kfp
from kfp import dsl
from google_cloud_pipeline_components.v1.custom_job import create_custom_training_job_from_component
# Create a Python component
@dsl.component
def my_python_component():
import time
time.sleep(1)
# Convert the above component into a custom training job
custom_training_job = create_custom_training_job_from_component(
my_python_component,
display_name = 'DISPLAY_NAME',
machine_type = 'MACHINE_TYPE',
accelerator_type='ACCELERATOR_TYPE',
accelerator_count='ACCELERATOR_COUNT',
boot_disk_type: 'BOOT_DISK_TYPE',
boot_disk_size_gb: 'BOOT_DISK_SIZE',
network: 'NETWORK',
reserved_ip_ranges: 'RESERVED_IP_RANGES',
nfs_mounts: 'NFS_MOUNTS'
)
# Define a pipeline that runs the custom training job
@dsl.pipeline(
name="resource-spec-request",
description="A simple pipeline that requests a Google Cloud machine resource",
pipeline_root='PIPELINE_ROOT',
)
def pipeline():
training_job_task = custom_training_job(
project='PROJECT_ID',
location='LOCATION',
).set_display_name('training-job-task')
Reemplaza lo siguiente:
DISPLAY_NAME: El nombre del trabajo personalizado. Si no especificas el nombre, se usa el nombre del componente de forma predeterminada.
MACHINE_TYPE: El tipo de máquina para ejecutar el trabajo personalizado, por ejemplo, e2-standard-4. Para obtener más información sobre los tipos de máquinas, consulta Tipos de máquinas. Si especificaste una TPU como el accelerator_type, configura esto como cloud-tpu.
Para obtener más información, consulta la referencia del parámetro machine_type.
ACCELERATOR_TYPE: Es el tipo de acelerador que se conecta a la máquina. Para obtener más información sobre las GPU disponibles y cómo configurarlas, consulta GPU. Para obtener más información sobre los tipos de TPU disponibles y cómo configurarlos, consulta TPU.
Para obtener más información, consulta la referencia del parámetro accelerator_type.
ACCELERATOR_COUNT: La cantidad de aceleradores adjuntos a la máquina que ejecuta el trabajo personalizado. Si especificas el tipo de acelerador, el recuento de aceleradores se establece en 1 de forma predeterminada.
NETWORK: si el trabajo personalizado realiza un intercambio de tráfico con una red
de Compute Engine que tiene configurado el acceso privado a servicios, especifica el nombre completo de la red. Para obtener más información, consulta la referencia del parámetro network.
RESERVED_IP_RANGES: una lista de nombres para los rangos de IP reservados
en la red de VPC que se usa para implementar el trabajo personalizado.
Para obtener más información, consulta la referencia del parámetro reserved_ip_ranges.
NFS_MOUNTS: una lista de recursos de activación de NFS en formato de diccionario JSON.
Para obtener más información, consulta la referencia del parámetro nfs_mounts.
PIPELINE_ROOT: Especifica un URI de Cloud Storage al que pueda acceder la cuenta de servicio de tus canalizaciones. Los artefactos de las ejecuciones de tus canalizaciones se almacenan en la raíz de la canalización.
PROJECT_ID: Es el proyecto de Google Cloud en el que se ejecuta esta canalización.
LOCATION: La ubicación o región en la que se ejecuta esta canalización.
[[["Fácil de comprender","easyToUnderstand","thumb-up"],["Resolvió mi problema","solvedMyProblem","thumb-up"],["Otro","otherUp","thumb-up"]],[["Difícil de entender","hardToUnderstand","thumb-down"],["Información o código de muestra incorrectos","incorrectInformationOrSampleCode","thumb-down"],["Faltan la información o los ejemplos que necesito","missingTheInformationSamplesINeed","thumb-down"],["Problema de traducción","translationIssue","thumb-down"],["Otro","otherDown","thumb-down"]],["Última actualización: 2024-07-05 (UTC)"],[],[],null,["# Learn how to request Google Cloud machine resources in Vertex AI Pipelines\n\nYou can run your Python component on Vertex AI Pipelines by using Google Cloud-specific machine resources offered by Vertex AI custom training.\n\nYou can use the [`create_custom_training_job_from_component`](https://google-cloud-pipeline-components.readthedocs.io/en/google-cloud-pipeline-components-2.19.0/api/v1/custom_job.html#v1.custom_job.create_custom_training_job_from_component) method from the [Google Cloud Pipeline Components](/vertex-ai/docs/pipelines/gcpc-list) to transform a Python component into a Vertex AI custom training job. [Learn how to create a custom job](/vertex-ai/docs/training/create-custom-job).\n\nCreate a custom training job from a component using Vertex AI Pipelines\n-----------------------------------------------------------------------\n\nThe following sample shows how to use the [`create_custom_training_job_from_component`](https://google-cloud-pipeline-components.readthedocs.io/en/google-cloud-pipeline-components-2.19.0/api/v1/custom_job.html#v1.custom_job.create_custom_training_job_from_component) method to transform a Python component into a custom training job with user-defined Google Cloud machine resources, and then run the compiled pipeline on Vertex AI Pipelines: \n\n\n import kfp\n from kfp import dsl\n from google_cloud_pipeline_components.v1.custom_job import create_custom_training_job_from_component\n\n # Create a Python component\n @dsl.component\n def my_python_component():\n import time\n time.sleep(1)\n\n # Convert the above component into a custom training job\n custom_training_job = create_custom_training_job_from_component(\n my_python_component,\n display_name = '\u003cvar translate=\"no\"\u003eDISPLAY_NAME\u003c/var\u003e',\n machine_type = '\u003cvar translate=\"no\"\u003eMACHINE_TYPE\u003c/var\u003e',\n accelerator_type='\u003cvar translate=\"no\"\u003eACCELERATOR_TYPE\u003c/var\u003e',\n accelerator_count='\u003cvar translate=\"no\"\u003eACCELERATOR_COUNT\u003c/var\u003e',\n boot_disk_type: '\u003cvar translate=\"no\"\u003eBOOT_DISK_TYPE\u003c/var\u003e',\n boot_disk_size_gb: '\u003cvar translate=\"no\"\u003eBOOT_DISK_SIZE\u003c/var\u003e',\n network: '\u003cvar translate=\"no\"\u003eNETWORK\u003c/var\u003e',\n reserved_ip_ranges: '\u003cvar translate=\"no\"\u003eRESERVED_IP_RANGES\u003c/var\u003e',\n nfs_mounts: '\u003cvar translate=\"no\"\u003eNFS_MOUNTS\u003c/var\u003e'\n persistent_resource_id: '\u003cvar translate=\"no\"\u003ePERSISTENT_RESOURCE_ID\u003c/var\u003e'\n )\n\n # Define a pipeline that runs the custom training job\n @dsl.pipeline(\n name=\"resource-spec-request\",\n description=\"A simple pipeline that requests a Google Cloud machine resource\",\n pipeline_root='\u003cvar translate=\"no\"\u003ePIPELINE_ROOT\u003c/var\u003e',\n )\n def pipeline():\n training_job_task = custom_training_job(\n project='\u003cvar translate=\"no\"\u003ePROJECT_ID\u003c/var\u003e',\n location='\u003cvar translate=\"no\"\u003eLOCATION\u003c/var\u003e',\n ).set_display_name('training-job-task')\n\nReplace the following:\n\n- \u003cvar translate=\"no\"\u003eDISPLAY_NAME\u003c/var\u003e: The name of the custom job. If you don't specify the name, the component name is used, by default.\n\n- \u003cvar translate=\"no\"\u003eMACHINE_TYPE\u003c/var\u003e: The type of the machine for running the custom job---for example, `e2-standard-4`. For more information about machine types, see [Machine types](/vertex-ai/docs/training/configure-compute#machine-types). If you specified a TPU as the `accelerator_type`, set this to `cloud-tpu`.\n For more information, see the [`machine_type` parameter reference](https://google-cloud-pipeline-components.readthedocs.io/en/google-cloud-pipeline-components-2.19.0/api/v1/custom_job.html#v1.custom_job.create_custom_training_job_from_component.machine_type).\n\n- \u003cvar translate=\"no\"\u003eACCELERATOR_TYPE\u003c/var\u003e: The type of accelerator attached to the machine. For more information about the available GPUs and how to configure them, see [GPUs](/vertex-ai/docs/training/configure-compute#specifying_gpus). For more information about the available TPU types and how to configure them, see [TPUs](/vertex-ai/docs/training/configure-compute#tpu).\n For more information, see the [`accelerator_type` parameter reference](https://google-cloud-pipeline-components.readthedocs.io/en/google-cloud-pipeline-components-2.19.0/api/v1/custom_job.html#v1.custom_job.create_custom_training_job_from_component.accelerator_type).\n\n- \u003cvar translate=\"no\"\u003eACCELERATOR_COUNT\u003c/var\u003e: The number of accelerators attached to the machine running the custom job. If you specify the accelerator type, the accelerator count is set to `1`, by default.\n\n- \u003cvar translate=\"no\"\u003eBOOT_DISK_TYPE\u003c/var\u003e: The type of boot disk.\n For more information, see the [`boot_disk_type` parameter reference](https://google-cloud-pipeline-components.readthedocs.io/en/google-cloud-pipeline-components-2.19.0/api/v1/custom_job.html#v1.custom_job.create_custom_training_job_from_component.boot_disk_type).\n\n- \u003cvar translate=\"no\"\u003eBOOT_DISK_SIZE\u003c/var\u003e: The size of the boot disk in GB.\n For more information, see the [`boot_disk_size_gb` parameter reference](https://google-cloud-pipeline-components.readthedocs.io/en/google-cloud-pipeline-components-2.19.0/api/v1/custom_job.html#v1.custom_job.create_custom_training_job_from_component.boot_disk_size_gb).\n\n- \u003cvar translate=\"no\"\u003eNETWORK\u003c/var\u003e: If the custom job is peered to a Compute Engine\n network that has private services access configured, specify the full name of the network. For more information, see the [`network` parameter reference](https://google-cloud-pipeline-components.readthedocs.io/en/google-cloud-pipeline-components-2.19.0/api/v1/custom_job.html#v1.custom_job.create_custom_training_job_from_component.network).\n\n- \u003cvar translate=\"no\"\u003eRESERVED_IP_RANGES\u003c/var\u003e: A list of names for the reserved IP ranges\n under the VPC network used to deploy the custom job.\n For more information, see the [`reserved_ip_ranges` parameter reference](https://google-cloud-pipeline-components.readthedocs.io/en/google-cloud-pipeline-components-2.19.0/api/v1/custom_job.html#v1.custom_job.create_custom_training_job_from_component.reserved_ip_ranges).\n\n- \u003cvar translate=\"no\"\u003eNFS_MOUNTS\u003c/var\u003e: A list of NFS mount resources in JSON dict format.\n For more information, see the [`nfs_mounts` parameter reference](https://google-cloud-pipeline-components.readthedocs.io/en/google-cloud-pipeline-components-2.19.0/api/v1/custom_job.html#v1.custom_job.create_custom_training_job_from_component.nfs_mounts).\n\n- \u003cvar translate=\"no\"\u003ePERSISTENT_RESOURCE_ID\u003c/var\u003e (preview): The ID of the persistent\n resource to run the pipeline. If you specify\n a persistent resource, the pipeline runs on existing machines\n associated to the persistent resource, instead of on-demand and short-lived\n machine resources. Note that the network and CMEK configuration for the\n pipeline must match the configuration specified for the persistent resource.\n For more information about persistent resources and how to create them, see\n [Create a persistent resource](/vertex-ai/docs/training/persistent-resource-create#create-persistent-resource-gcloud).\n\n- \u003cvar translate=\"no\"\u003ePIPELINE_ROOT\u003c/var\u003e: Specify a Cloud Storage URI that your pipelines service account can access. The artifacts of your pipeline runs are stored within the pipeline root.\n\n- \u003cvar translate=\"no\"\u003ePROJECT_ID\u003c/var\u003e: The Google Cloud project that this pipeline runs in.\n\n- \u003cvar translate=\"no\"\u003eLOCATION\u003c/var\u003e: The location or region that this pipeline runs in.\n\nAPI Reference\n-------------\n\nFor a complete list of arguments supported by the [`create_custom_training_job_from_component`](https://google-cloud-pipeline-components.readthedocs.io/en/google-cloud-pipeline-components-2.19.0/api/v1/custom_job.html#v1.custom_job.create_custom_training_job_from_component) method, see the [Google Cloud Pipeline Components SDK Reference](https://google-cloud-pipeline-components.readthedocs.io/en/google-cloud-pipeline-components-2.19.0/api/v1/index.html)."]]