Migrate from user-managed notebooks to Vertex AI Workbench instances

Migrate from user-managed notebooks to Vertex AI Workbench instances

Vertex AI Workbench user-managed notebooks is deprecated. On January 30, 2025, support for user-managed notebooks will end and the ability to create user-managed notebooks instances will be removed. Existing instances will continue to function but patches, updates, and upgrades won't be available. To continue using Vertex AI Workbench, complete the steps on this page to migrate your user-managed notebooks instances to Vertex AI Workbench instances.

This page describes how to migrate from a user-managed notebooks instance to a Vertex AI Workbench instance. You can migrate by using the Vertex AI Workbench migration tool or migrate your instance's data and files manually.

Overview of the migration tool

Vertex AI Workbench provides a migration tool for migrating from a user-managed notebooks instance to a Vertex AI Workbench instance.

The migration tool creates a Vertex AI Workbench instance with a configuration similar to the user-managed notebooks instance that you want to migrate. For example, the migration tool creates an instance that has the same or similar machine type, network configuration, idle shutdown settings, and other specifications. Then, the files on your user-managed notebooks instance's data disk are copied to the Vertex AI Workbench instance.

Vertex AI Workbench doesn't delete or change your user-managed notebooks instance, so after migration you can continue to use it. If you don't need the user-managed notebooks instance anymore, delete it to avoid further charges for that instance.

Billing

If your user-managed notebooks instance uses Extreme Persistent Disks, then your migration generates charges for I/O operations. See "Extreme provisioned IOPS" in the Persistent Disk and Hyperdisk pricing section of Disk pricing.

After migration, the user-managed notebooks instance still exists and generates charges as before. If you don't need the user-managed notebooks instance anymore, delete it to avoid further charges for that instance.

Default migration tool behaviors

The Vertex AI Workbench migration tool attempts to migrate your user-managed notebooks instance to a Vertex AI Workbench instance with matching specifications. When a specification in your user-managed notebooks instance isn't available in Vertex AI Workbench instances, Vertex AI Workbench uses a default specification when possible. When the migration tool can't migrate a specification of your user-managed notebooks instance, it doesn't migrate the instance.

The following table lists some of the key default migration behaviors for the migration tool.

Category	user-managed notebooks specification	Migration result
OS	Any Ubuntu version	Debian 11
	Any Debian version	Debian 11
Framework	Any CUDA version	CUDA 11.3
	Any Python version	Python 3.10
	Any PyTorch version	PyTorch 1.13
	Any TensorFlow version	TensorFlow 2.11
	Any R version	Not migrated; see Add a conda environment
	Any local PySpark version	Not migrated; see Add a conda environment
	Any XGBoost version	Not migrated; see Add a conda environment
	Any Kaggle Python version	Not migrated; see Add a conda environment
	Any Jax version	Not migrated; see Add a conda environment
	Any Apache Beam version	Not migrated; see Add a conda environment
Machine type	A supported machine type	Identical machine type
	An unsupported machine type	e2-standard-4
Accelerators	Supported accelerators	Identical accelerators
	Unsupported accelerators	Migration doesn't include accelerators
Setting	Idle shutdown	Migrated
	Delete to trash	Migrated
	nbconvert	Migrated
	File downloading	Migrated
	Terminal access	Migrated
Other	Identity and Access Management permissions	Migrated, though new permissions might be required to use the Vertex AI Workbench instance
	Access mode	Migrated
	Network	Migrated
	Post-startup script	When using the Google Cloud console, the instance is migrated without the post-startup script; to migrate the instance with the post-startup script, use the Google Cloud CLI or REST API to specify the PostStartupScriptOption option
	Dataproc Hub	Not migrated; must migrate manually

Specifying the post-startup script

User-managed notebooks instances that use a post-startup script must be migrated to an instance with the PostStartupScriptOption option specified. Use this option to indicate whether you want to skip or rerun the post-startup script in your new Vertex AI Workbench instance.

Specifying the PostStartupScriptOption option isn't supported in the Google Cloud console. To specify the PostStartupScriptOption option when you migrate your user-managed notebooks instance, you must use the Google Cloud CLI or REST API.

Before you begin

1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

2. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   Go to project selector

3. Make sure that billing is enabled for your Google Cloud project.

4. Enable the Notebooks API.

   Enable the API

5. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   Go to project selector

6. Make sure that billing is enabled for your Google Cloud project.

7. Enable the Notebooks API.

   Enable the API

8. If you haven't already, create a user-managed notebooks instance.

Required roles

To ensure that your user account has the necessary permissions to migrate a user-managed notebooks instance to a Vertex AI Workbench instance, ask your administrator to grant your user account the Notebooks Runner (roles/notebooks.runner) IAM role on the project. For more information about granting roles, see Manage access to projects, folders, and organizations.

This predefined role contains the permissions required to migrate a user-managed notebooks instance to a Vertex AI Workbench instance. To see the exact permissions that are required, expand the Required permissions section:

Required permissions

The following permissions are required to migrate a user-managed notebooks instance to a Vertex AI Workbench instance:

• notebooks.instances.create
• notebooks.instances.get

Your administrator might also be able to give your user account these permissions with custom roles or other predefined roles.

Pre-migration check

Before you migrate, check your user-managed notebooks instance's migration eligibility by listing your instances and checking the output for any migration warnings or errors.

List your instances

To list your user-managed notebooks instances that aren't migrated yet, use the projects.locations.instances.list method with the filter migrated:false. You can list them by using the gcloud CLI or REST API:

gcloud

Before using any of the command data below, make the following replacements:

• PROJECT_ID: Your project ID
• LOCATION: The region where your user-managed notebooks instance is located, or use - to list instances from all regions

Execute the following command:

Linux, macOS, or Cloud Shell

Note: Ensure you have initialized the Google Cloud CLI with authentication and a project by running either gcloud init; or gcloud auth login and gcloud config set project.

gcloud notebooks instances list --project=PROJECT_ID \
    --location=LOCATION --filter=migrated:false --format=default

Windows (PowerShell)

Note: Ensure you have initialized the Google Cloud CLI with authentication and a project by running either gcloud init; or gcloud auth login and gcloud config set project.

gcloud notebooks instances list --project=PROJECT_ID `
    --location=LOCATION --filter=migrated:false --format=default

Windows (cmd.exe)

Note: Ensure you have initialized the Google Cloud CLI with authentication and a project by running either gcloud init; or gcloud auth login and gcloud config set project.

gcloud notebooks instances list --project=PROJECT_ID ^
    --location=LOCATION --filter=migrated:false --format=default

REST

Before using any of the request data, make the following replacements:

• PROJECT_ID: Your project ID
• LOCATION: The region where your user-managed notebooks instance is located, or use - to list instances from all regions

HTTP method and URL:

GET https://notebooks.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances?filter=migrated:false

To send your request, choose one of these options:

curl

Note: The following command assumes that you have logged in to the gcloud CLI with your user account by running gcloud init or gcloud auth login , or by using Cloud Shell, which automatically logs you into the gcloud CLI . You can check the currently active account by running gcloud auth list.

Execute the following command:

curl -X GET \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     "https://notebooks.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances?filter=migrated:false"

PowerShell

Note: The following command assumes that you have logged in to the gcloud CLI with your user account by running gcloud init or gcloud auth login . You can check the currently active account by running gcloud auth list.

Execute the following command:

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method GET `
    -Headers $headers `
    -Uri "https://notebooks.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances?filter=migrated:false" | Select-Object -Expand Content

Check the output for warnings or errors

If migration warnings or errors are detected, the output of the projects.locations.instances.list method includes this information.

Warnings appear when specific components in your user-managed notebooks instance's configuration won't migrate to the same specification in a Vertex AI Workbench instance. For example, if your user-managed notebooks instance uses an unsupported accelerator, a warning appears in the output. In this case, the instance is migrated without any accelerators. You can attach accelerators after migration. Review the warnings in the output, consider the migration tool's default behaviors, and assess whether the migration tool is acceptable for your migration.

One or more errors in the output means that you can't migrate the user-managed notebooks instance by using the migration tool. You must migrate the instance manually.

For more information about migration warnings and errors, see warnings and errors in the InstanceMigrationEligibility documentation.

Migrate by using the migration tool

You can migrate your user-managed notebooks instance by using the Google Cloud console, the gcloud CLI, or REST API.

Console

1. In the Google Cloud console, go to the User-managed notebooks page.

   Go to User-managed notebooks

2. Click the Migrate button.

3. On the Migrate user-managed notebooks to instances page, on the Ready tab, select the instance that you want to migrate.

4. Click Migrate.

5. After the migration is finished, go to the Instances page to view your new Vertex AI Workbench instance.

   Go to Instances

gcloud

Before using any of the command data below, make the following replacements:

• PROJECT_ID: Your project ID
• LOCATION: The region where your user-managed notebooks instance is located
• INSTANCE_ID: The ID of the user-managed notebooks instance
• POST_STARTUP_SCRIPT_OPTION: Optional: One of the post-startup script options

Execute the following command:

Linux, macOS, or Cloud Shell

Note: Ensure you have initialized the Google Cloud CLI with authentication and a project by running either gcloud init; or gcloud auth login and gcloud config set project.

gcloud notebooks instances migrate RUNTIME_ID \
    --project=PROJECT_ID \
    --location=LOCATION \
    --post-startup-script-option=POST_STARTUP_SCRIPT_OPTION

Windows (PowerShell)

Note: Ensure you have initialized the Google Cloud CLI with authentication and a project by running either gcloud init; or gcloud auth login and gcloud config set project.

gcloud notebooks instances migrate RUNTIME_ID `
    --project=PROJECT_ID `
    --location=LOCATION `
    --post-startup-script-option=POST_STARTUP_SCRIPT_OPTION

Windows (cmd.exe)

Note: Ensure you have initialized the Google Cloud CLI with authentication and a project by running either gcloud init; or gcloud auth login and gcloud config set project.

gcloud notebooks instances migrate RUNTIME_ID ^
    --project=PROJECT_ID ^
    --location=LOCATION ^
    --post-startup-script-option=POST_STARTUP_SCRIPT_OPTION

REST

Before using any of the request data, make the following replacements:

• PROJECT_ID: Your project ID
• LOCATION: The region where your user-managed notebooks instance is located
• INSTANCE_ID: The ID of the user-managed notebooks instance
• POST_STARTUP_SCRIPT_OPTION: Optional: One of the post-startup script options

HTTP method and URL:

POST https://notebooks.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances/INSTANCE_ID:migrate

Request JSON body:

{
  "postStartupScriptOption": (POST_STARTUP_SCRIPT_OPTION)
}

To send your request, choose one of these options:

curl

Note: The following command assumes that you have logged in to the gcloud CLI with your user account by running gcloud init or gcloud auth login , or by using Cloud Shell, which automatically logs you into the gcloud CLI . You can check the currently active account by running gcloud auth list.

Save the request body in a file named request.json, and execute the following command:

curl -X POST \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     -H "Content-Type: application/json; charset=utf-8" \
     -d @request.json \
     "https://notebooks.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances/INSTANCE_ID:migrate"

PowerShell

Note: The following command assumes that you have logged in to the gcloud CLI with your user account by running gcloud init or gcloud auth login . You can check the currently active account by running gcloud auth list.

Save the request body in a file named request.json, and execute the following command:

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method POST `
    -Headers $headers `
    -ContentType: "application/json; charset=utf-8" `
    -InFile request.json `
    -Uri "https://notebooks.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/instances/INSTANCE_ID:migrate" | Select-Object -Expand Content

Migrate manually

To migrate your instance to a Vertex AI Workbench instance manually, consider using the following methods:

• Use Cloud Storage and the terminal: Copy your data and files to Cloud Storage and then to another instance by using the terminal.

• Use GitHub: Copy your data and files to a GitHub repository by using the Git extension for JupyterLab.

This guide describes how to migrate data and files by using Cloud Storage and the terminal.

Requirements

You must have terminal access to your user-managed notebooks instance. Terminal access is manually set when you create an instance. The terminal access setting cannot be changed after the instance is created.

Migrate manually by using Cloud Storage and the terminal

To migrate data and files to a new Vertex AI Workbench instance by using Cloud Storage and the terminal, do the following.

1. Create a Cloud Storage bucket in the same project where your user-managed notebooks instance is located.

2. In that same project, Create a Vertex AI Workbench instance to migrate your data to. When you create this instance:

   • Enable terminal access.
   • Specify the machine type, network, and other characteristics to match what you need.

3. In your user-managed notebooks instance's JupyterLab interface, select File > New > Terminal to open a terminal window.

4. Use the gcloud CLI to copy your user data to a Cloud Storage bucket. The following example command copies all of the files from your instance's /home/jupyter/ directory to a directory in a Cloud Storage bucket.

   gcloud storage cp /home/jupyter/* gs://BUCKET_NAMEPATH --recursive

   Replace the following:

   • BUCKET_NAME: The name of your Cloud Storage bucket
   • PATH: The path to the directory where you want to copy your files, for example: /copy/jupyter/

5. In your new Vertex AI Workbench instance's JupyterLab interface, select File > New > Terminal to open a terminal window.

6. Use the gcloud CLI to copy your data to the new instance. The following example command copies all of the files from a Cloud Storage directory to the your new instance's /home/jupyter/ directory.

   gcloud storage cp gs://BUCKET_NAMEPATH* /home/jupyter/

Confirm the migration

After the migration, the original user-managed notebooks instance continues to work as before. Confirm that your migration was a success before you delete the original instance.

Delete the user-managed notebooks instance

If you don't need the user-managed notebooks instance that you migrated from, delete it to avoid further charges for that instance.

1. In the Google Cloud console, go to the User-managed notebooks page.

   Go to User-managed notebooks

2. Select the instance that you want to delete.

3. Click  Delete. (Depending on the size of your window, the Delete button might be in the  options menu.)

4. To confirm, click Delete.

Troubleshoot

To find methods for diagnosing and resolving migration issues, see Troubleshooting Vertex AI Workbench.

What's next

• Learn more about Vertex AI Workbench instances.