Connecting to AI Platform Pipelines using the Kubeflow Pipelines SDK

You can use the Kubeflow Pipelines SDK to manage and automate activity on your AI Platform Pipelines cluster, such as:

  • Loading pipelines to your AI Platform Pipelines cluster
  • Running pipelines
  • Retrieving information about pipelines and runs

This guide describes how to connect to your AI Platform Pipelines cluster using the Kubeflow Pipelines SDK.

Before you start

Before you connect to your cluster using the Kubeflow Pipelines SDK, you must first set up your AI Platform Pipelines cluster and install the Kubeflow Pipelines SDK.

  1. Follow the guide to setting up your AI Platform Pipelines cluster.
  2. To install the Kubeflow Pipelines SDK in your Jupyter notebook or Python client, follow the Kubeflow guide to installing the Kubeflow Pipelines SDK.

Using the Kubeflow Pipelines SDK to connect to an AI Platform Pipelines cluster

Use the following instructions to connect to your AI Platform Pipelines cluster using the Kubeflow Pipelines SDK.

  1. Open AI Platform Pipelines in the Google Cloud Console.

    Go to AI Platform Pipelines

  2. Click Open pipelines dashboard for your Kubeflow Pipelines cluster. The Kubeflow Pipelines user interface opens in a new tab.

  3. Find the hostname in the URL of the Kubeflow Pipelines dashboard. The hostname is the portion of the URL between https:// and /#/start, and should match the pattern *.pipelines.googleusercontent.com.

  4. Use the Kubeflow Pipelines SDK to connect to your AI Platform Pipelines cluster from a Jupyter notebook or Python client.

    import kfp
client = kfp.Client(host='example.com')

    Replace the example.com hostname, with the hostname for your cluster.

  5. Now you can use the Kubeflow Pipelines SDK to interact with your cluster. For example, the following command lists your cluster's pipelines.

    client.list_pipelines()

What's next