Run notebook files with the executor
This page describes how to use the executor in a Vertex AI Workbench managed notebooks instance to run notebook files as a one-time execution and on a schedule.
Overview
The executor lets you submit a notebook (ipynb) file to run on Vertex AI custom training. You can set parameter values for each execution of a notebook file. You can also run a notebook file on a recurring schedule. After the execution completes, you can view the execution results and share them with others.
When a notebook file is submitted to Vertex AI custom training, Vertex AI creates a new custom training job that executes your notebook file following the lifecycle of a training job.
Requirements for notebook code run by the executor
When you write notebook code to run in the executor, keep in mind that the code will run in a tenant project separate from your managed notebooks instance's project. This section describes how this affects your code when it is run in the executor.
Ensure package installations are available to the executor
If your notebook depends on package installations that are not already included in the managed notebooks kernel that you are using, make sure your packages are available to your notebook code in the executor in one of the following ways:
Use a custom container that already has the package installed, and then execute your notebook on that custom container. See the requirements for using a custom container with the executor.
Install the package within your notebook file's code. The package is installed every time the notebook file is executed, but this ensures that the package is available on the container that you select when you execute your notebook.
Use explicit project selection
When you access resources through code run by the executor, the executor might not connect to the correct Google Cloud project. If you encounter permission errors, connecting to the wrong project might be the problem.
This problem occurs because the executor does not run your code directly in your managed notebooks instance's Google Cloud project. Instead, the executor runs your code in Vertex AI custom training within a tenant project managed by Google. Therefore, don't try to infer a project ID from the environment in your notebook code; specify project IDs explicitly.
If you don't want to hardcode a project ID in your code, you can
reference the CLOUD_ML_PROJECT_ID
environment variable.
Vertex AI
sets this environment variable in every custom training container to contain the
project number of the project where you initiated
custom training. Many Google Cloud tools can accept a project
number wherever they take a project ID.
For example, if you want to use the Python Client for Google BigQuery to access a BigQuery table in the same project, then do not infer the project in your notebook code:
Implicit project selection
from google.cloud import bigquery
client = bigquery.Client()
Instead use code that explicitly selects a project:
Explicit project selection
import os
from google.cloud import bigquery
project_number = os.environ["CLOUD_ML_PROJECT_ID"]
client = bigquery.Client(project=project_number)
Authenticate access using service accounts
By default, your managed notebooks instance can have access to resources that exist in the same project. Therefore, when you run your notebook file's code manually, these resources do not need additional authentication. However, because the executor runs in a separate tenant project, it does not have the same default access.
Also, the executor cannot use end-user credentials to authenticate access
to resources, for example, the
gcloud auth login
command.
To resolve these issues, in your notebook file's code, authenticate access to resources through a service account.
Then when you create an execution or schedule, specify the service account.
For example, while you create an execution, complete these steps:
In the Submit notebooks to Executor dialog, expand Advanced options.
In the Identity and API access section, clear the check mark next to Use Vertex AI Training's default service account and enter the specific service account to use.
See the full list of steps for creating an execution.
Requirements when using a custom container
You can use the executor to run notebook code on a custom container.
Your custom container must include the nbexecutor
extension, which
enables the executor to run notebook code as
a Vertex AI custom training job.
To ensure that your custom container has the nbexecutor
extension,
you can modify one of the
Deep Learning Containers container images to create a
derivative container image.
Deep Learning Containers images
include the nbexecutor
extension.
Before you begin
- Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.
-
In the Google Cloud console, on the project selector page, select or create a Google Cloud project.
-
Make sure that billing is enabled for your Google Cloud project.
-
Enable the Notebooks and Vertex AI APIs.
-
In the Google Cloud console, on the project selector page, select or create a Google Cloud project.
-
Make sure that billing is enabled for your Google Cloud project.
-
Enable the Notebooks and Vertex AI APIs.
- If you haven't already, create a managed notebooks instance.
Required roles
To ensure that your instance's service account has the necessary permissions to interact with the Vertex AI Workbench executor, ask your administrator to grant your instance's service account the following IAM roles on the project:
-
Notebooks Viewer (
roles/notebooks.viewer
) -
Vertex AI User (
roles/aiplatform.user
) -
Storage Admin (
roles/storage.admin
)
For more information about granting roles, see Manage access to projects, folders, and organizations.
Your administrator might also be able to give your instance's service account the required permissions through custom roles or other predefined roles.
Open JupyterLab
To open JupyterLab and prepare a notebook file to run, complete the following steps.
Upload a notebook (ipynb) file, open an existing file, or open a new notebook file and add the code that you want to run to the notebook.
Make sure your notebook file's code meets the requirements for using the executor.
Create an execution
To create an execution that runs your notebook file, complete the following steps. These steps cover both scheduling executions and creating a one-time execution.
In your managed notebooks instance's JupyterLab user interface, open the notebook file that you want to run.
Click the Execute button.
In the Submit notebooks to Executor dialog, in the Execution name field, enter a name for your execution.
Select a Machine type and Accelerator type.
Select an Environment.
In the Type field, select One-time execution or select Schedule-based recurring executions and complete the dialog for scheduling executions.
In Advanced options, select the Region where you want to run your notebook.
In the Cloud Storage bucket field, select an available Cloud Storage bucket or enter a name for a new bucket and click Create and select. The executor stores your notebook output in this Cloud Storage bucket.
Optional: in the Notebook parameterization section, in the Input parameters text box, add notebook parameters separated by commas, for example
optimizer="SGD",learning_rate=0.01
.Learn more about how to use notebook parameters.
Optional: in the Identity and API access section, select Use Vertex AI Training's default service account or clear the check mark and enter a specific service account to use.
Optional: in the Networking section, specify a Virtual Private Cloud network. Using a VPC network for your execution requires a private services access connection.
Click Submit.
One-time executions begin immediately. Scheduled executions run automatically on the schedule that you set.
In the Google Cloud console, on the Vertex AI Workbench page, you can view your completed executions on the Executions tab and view your schedules on the Schedules tab.
View, share, and import an executed notebook file
By using your managed notebooks instance's JupyterLab user interface, you can view an executed notebook's output, share the results with others, and import the executed notebook file into JupyterLab.
View a notebook execution's results
You can view a notebook execution's results in the Google Cloud console or in the JupyterLab user interface.
Console
In the Google Cloud console, go to the Vertex AI Workbench page and click the Executions tab.
Select the Region that contains your results.
Next to the execution that you want to view, click View result.
The result opens in a new browser tab.
JupyterLab
In JupyterLab's navigation menu, click the Notebook Executor button.
Click the Executions tab.
Under the execution that you want to view, click View result.
The result opens in a new browser tab.
Share a notebook execution's results
You can share execution results by providing access to the Cloud Storage bucket that contains your notebook execution. Providing this access also grants users access to any other resources in the same Cloud Storage bucket. To share execution results, complete the following steps.
Console
In the Google Cloud console, go to the Vertex AI Workbench page and click the Executions tab.
Select the Region that contains the execution.
Next to the execution that you want to share, click the
Share button.Follow the directions in the dialog to grant users access to the Cloud Storage bucket that contains your notebook execution.
JupyterLab
In your managed notebooks instance's JupyterLab user interface, in the navigation menu, click the Notebook Executor button.
Click the Executions tab.
Next to the execution that you want to share, click the
options menu, and select Share execution result.Follow the directions in the dialog to grant users access to the Cloud Storage bucket that contains your notebook execution.
Import an executed notebook into JupyterLab
To import an executed notebook into JupyterLab, complete the following steps.
In your managed notebooks instance's JupyterLab user interface, in the navigation menu, click the Notebook Executor button.
Click the Executions tab.
Select the Region that contains your executed notebook.
Next to the execution that you want to import, click the
options menu, and select Import executed notebook.Select the kernel that you want to open the notebook.
The executor opens the executed notebook file in JupyterLab and stores this notebook file in the JupyterLab File Browser in a folder named imported_notebook_jobs.
View or delete a schedule
You can view and delete schedules by using either the Google Cloud console or your managed notebooks instance's JupyterLab user interface.
View a schedule
View a schedule to see the frequency settings of the schedule or to view results of your notebook executions.
Console
In the Google Cloud console, go to the Vertex AI Workbench page and click the Schedules tab.
Select the Region that contains your schedule.
Click a schedule name to open the Schedule details page.
Next to an execution name, click View result to open the executed notebook file. The executor opens your result in a new browser tab.
JupyterLab
In your managed notebooks instance's JupyterLab user interface, in the navigation menu, click the Notebook Executor button.
Click the Schedules tab.
To view the latest execution, under the execution that you want to view, click View latest execution result. The executor opens your result in a new browser tab.
To see all of the executions, click the name of the schedule. The executor opens the Schedule details page in the Google Cloud console.
Next to an execution name, click View result to open the executed notebook file. The executor opens your result in a new browser tab.
Delete a schedule
Deleting a schedule does not delete the executions that were generated from that schedule.
Console
In the Google Cloud console, go to the Vertex AI Workbench page and click the Schedules tab.
Select the Region that contains your schedule.
Select the schedule you want to delete.
Click
Delete.
JupyterLab
In your managed notebooks instance's JupyterLab user interface, in the navigation menu, click the Notebook Executor button.
Click the Schedules tab.
Click the name of the schedule that you want to delete. The executor opens the Schedule details page in the Google Cloud console.
Click
Delete.
Jobs on Vertex AI custom training
Because notebook executions are run on Vertex AI custom training, they are exposed as custom training jobs in Vertex AI. You can view these custom training jobs in the Google Cloud console, on the Custom jobs tab of the Vertex AI Training page. Learn more about how to work with Vertex AI custom training jobs.
What's next
Learn how to run notebook executions with parameters.
Learn more about Vertex AI custom training.