Introduction to Google Cloud Pipeline Components

The Google Cloud Pipeline Components (GCPC) SDK provides a set of prebuilt components that are production quality, consistent, performant, and easy to use in Vertex AI Pipelines. You can use Google Cloud Pipeline Components to perform ML tasks. For example, you can use these components to complete the following:

  • Create a new dataset and load different data types into the dataset (image, tabular, text, or video).
  • Export data from a dataset to Cloud Storage.
  • Use AutoML to train a model using image, tabular, text, or video data.
  • Run a custom training job using a custom container or a Python package.
  • Upload an existing model to Vertex AI for batch prediction.
  • Create a new endpoint and deploy a model to it for online predictions.

Additionally, these prebuilt Google Cloud Pipeline Components are supported in Vertex AI Pipelines and offer the following benefits:

  • Easier debugging: Show the underlying resources launched from the component for simplified debugging.
  • Standardized artifact types: Provide consistent interfaces to use standard artifact types for input and output. These standard artifacts are tracked in Vertex ML Metadata, making it easier for you to analyze the lineage of your pipeline's artifacts. For more details on artifact lineage, see Tracking the lineage of pipeline artifacts.
  • Cost efficiencies*: Vertex AI Pipelines optimize the execution of these components by launching the GCP resources, without having to launch the container. This reduces the startup latency and reduces the costs of the busy-waiting container.
* This feature applies to the following components only:
  • CustomTrainingJobOp
  • WaitGcpResourcesOp (for Dataflow)

What's next