Migrate from managed notebooks to Vertex AI Workbench instances

This page describes how to migrate from a managed notebooks instance to a Vertex AI Workbench instance. You can migrate by using the Vertex AI Workbench migration tool or migrate your instance's data and files manually.

Overview of the migration tool

Vertex AI Workbench provides a migration tool for migrating from a managed notebooks instance to a Vertex AI Workbench instance.

The migration tool creates a Vertex AI Workbench instance with a configuration similar to the managed notebooks instance that you want to migrate. For example, the migration tool creates an instance that has the same or similar machine type, network configuration, idle shutdown settings, and other specifications. Then, the files on your managed notebooks instance's data disk are copied to the Vertex AI Workbench instance.

Vertex AI Workbench doesn't delete or change your managed notebooks instance, so after migration you can continue to use it. If you don't need the managed notebooks instance anymore, delete it to avoid further charges for that instance.

Billing

If your managed notebooks instance uses Extreme Persistent Disks, then your migration generates charges for I/O operations. See "Extreme provisioned IOPS" in the Persistent Disk and Hyperdisk pricing section of Disk pricing.

After migration, the managed notebooks instance still exists and generates charges as before. If you don't need the managed notebooks instance anymore, delete it to avoid further charges for that instance.

Default migration tool behaviors

The Vertex AI Workbench migration tool attempts to migrate your managed notebooks instance to a Vertex AI Workbench instance with matching specifications. When a specification in your managed notebooks instance isn't available in Vertex AI Workbench instances, Vertex AI Workbench uses a default specification when possible. When the migration tool can't migrate a specification of your managed notebooks instance, it doesn't migrate the instance.

The following table lists some of the key default migration behaviors for the migration tool.

Category	Managed notebooks specification	Migration result
OS	Any Ubuntu version	Debian 11
OS	Any Debian version	Debian 11
Framework	Any CUDA version	CUDA 11.3
	Any Python version	Python 3.10
	Any PyTorch version	PyTorch 1.13
	Any TensorFlow version	TensorFlow 2.11
	Any R version	Not migrated; see Add a conda environment
	Any local PySpark version	Not migrated; see Add a conda environment
	Any XGBoost version	Not migrated; see Add a conda environment
	Any Kaggle Python version	Not migrated; see Add a conda environment
	Any Jax version	Not migrated; see Add a conda environment
	Any Apache Beam version	Not migrated; see Add a conda environment
Machine type	A supported machine type	Identical machine type
Machine type	An unsupported machine type	`e2-standard-4`
Accelerators	Supported accelerators	Identical accelerators
Accelerators	Unsupported accelerators	Migration doesn't include accelerators
Setting	Idle shutdown	Migrated
	Delete to trash	Migrated
	`nbconvert`	Migrated
	File downloading	Migrated
	Terminal access	Migrated
Other	Identity and Access Management permissions	Migrated, though new permissions might be required to use the Vertex AI Workbench instance
	Access mode	Migrated; instances that use the single user JupyterLab access mode must specify the `serviceAccount` option
	Network	Migrated; instances that use a Google-managed Virtual Private Cloud must specify the `network` and `subnet` options
	Post-startup script	When using the Google Cloud console, the instance is migrated without the post-startup script; to migrate the instance with the post-startup script, use the Google Cloud CLI or REST API to specify the `PostStartupScriptOption` option
	Dataproc Hub	Not migrated; must migrate manually

Specifying options

The following sections describe cases where specifying an option is required to migrate your managed notebooks instance to a Vertex AI Workbench instance.

Instances that use the single user access mode

Managed notebooks instances that use the single user access mode must be migrated to an instance with the serviceAccount option specified. The Vertex AI Workbench instance that you migrate to restricts access to JupyterLab to the single user, but it uses a service account to interact with Google Cloud services and APIs.

Instances that use a Google-managed VPC

Managed notebooks instances that use a Google-managed VPC must be migrated to an instance with the network and subnet options specified. The option to use a Google-managed VPC isn't supported in Vertex AI Workbench instances, so a different network must be specified.

Instances that use a post-startup script

Managed notebooks instances that use a post-startup script must be migrated to an instance with the PostStartupScriptOption option specified. Use this option to indicate whether you want to skip or rerun the post-startup script in your new Vertex AI Workbench instance.

Specifying the PostStartupScriptOption option isn't supported in the Google Cloud console. To specify the PostStartupScriptOption option when you migrate your managed notebooks instance, you must use the Google Cloud CLI or REST API.

Before you begin

Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

Go to project selector

Make sure that billing is enabled for your Google Cloud project.

Enable the Notebooks API.

Enable the API

In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

Go to project selector

Make sure that billing is enabled for your Google Cloud project.

Enable the Notebooks API.

Enable the API

If you haven't already, create a managed notebooks instance.

Required roles

To ensure that your user account has the necessary permissions to migrate a managed notebooks instance to a Vertex AI Workbench instance, ask your administrator to grant your user account the Notebooks Runner (roles/notebooks.runner) IAM role on the project. For more information about granting roles, see Manage access.

This predefined role contains the permissions required to migrate a managed notebooks instance to a Vertex AI Workbench instance. To see the exact permissions that are required, expand the Required permissions section:

Required permissions

The following permissions are required to migrate a managed notebooks instance to a Vertex AI Workbench instance:

notebooks.runtimes.create
notebooks.runtimes.get

Your administrator might also be able to give your user account these permissions with custom roles or other predefined roles.

Pre-migration check

Before you migrate, check your managed notebooks instance's migration eligibility by listing your instances and checking the output for any migration warnings or errors.

List your instances

To list your managed notebooks instances that aren't migrated yet, use the projects.locations.runtimes.list method with the filter migrated:false. You can list them by using the gcloud CLI or REST API:

gcloud

Before using any of the command data below, make the following replacements:

PROJECT_ID: Your project ID
LOCATION: The region where your managed notebooks instance is located, or use - to list instances from all regions

Execute the following command:

Linux, macOS, or Cloud Shell

gcloud notebooks runtimes list --project=PROJECT_ID \
    --location=LOCATION --filter=migrated:false --format=default

Windows (PowerShell)

gcloud notebooks runtimes list --project=PROJECT_ID `
    --location=LOCATION --filter=migrated:false --format=default

Windows (cmd.exe)

gcloud notebooks runtimes list --project=PROJECT_ID ^
    --location=LOCATION --filter=migrated:false --format=default

REST

Before using any of the request data, make the following replacements:

PROJECT_ID: Your project ID
LOCATION: The region where your managed notebooks instance is located, or use - to list instances from all regions

HTTP method and URL:

GET https://notebooks.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/runtimes?filter=migrated:false

To send your request, choose one of these options:

curl

Note: The following command assumes that you have logged in to the gcloud CLI with your user account by running gcloud init or gcloud auth login , or by using Cloud Shell, which automatically logs you into the gcloud CLI . You can check the currently active account by running gcloud auth list.

Execute the following command:

curl -X GET \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     "https://notebooks.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/runtimes?filter=migrated:false"

PowerShell

Note: The following command assumes that you have logged in to the gcloud CLI with your user account by running gcloud init or gcloud auth login . You can check the currently active account by running gcloud auth list.

Execute the following command:

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method GET `
    -Headers $headers `
    -Uri "https://notebooks.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/runtimes?filter=migrated:false" | Select-Object -Expand Content

Check the output for warnings or errors

If migration warnings or errors are detected, the output of the projects.locations.runtimes.list method includes this information.

Warnings appear when specific components in your managed notebooks instance's configuration won't migrate to the same specification in a Vertex AI Workbench instance. For example, if your managed notebooks instance uses an unsupported accelerator, a warning appears in the output. In this case, the instance is migrated without any accelerators. You can attach accelerators after migration. Review the warnings in the output, consider the migration tool's default behaviors, and assess whether the migration tool is acceptable for your migration.

One or more errors in the output means that you can't migrate the managed notebooks instance by using the migration tool. You must migrate the instance manually.

For more information about migration warnings and errors, see warnings and errors in the RuntimeMigrationEligibility documentation.

Migrate by using the migration tool

You can migrate your managed notebooks instance by using the Google Cloud console, the gcloud CLI, or REST API.

Console

In the Google Cloud console, go to the Managed notebooks page.

Go to Managed notebooks
Click the Migrate button. The Migrate managed notebooks to instances page opens.
To migrate instances that don't need options specified, click the Ready tab, select the instances that you want to migrate, and then click Migrate.
To migrate instances that need options specified, click the Needs input tab, select the instances that you want to migrate, and then click Migrate.
1. In the Provide input for migration dialog, specify a network and service account to use for the new Vertex AI Workbench instances that you selected.
2. Click Submit.
After your migrations are finished, go to the Instances page to view your new Vertex AI Workbench instances.

Go to Instances

gcloud

Before using any of the command data below, make the following replacements:

PROJECT_ID: Your project ID
LOCATION: The region where your managed notebooks instance is located
RUNTIME_ID: The ID of the managed notebooks instance
NETWORK: Optional: The network that you want to migrate the instance to
SUBNET: Optional: The subnet that you want to migrate the instance to
SERVICE_ACCOUNT: Optional: The email address of the service account that you want to use
POST_STARTUP_SCRIPT_OPTION: Optional: One of the post-startup script options

Execute the following command:

Linux, macOS, or Cloud Shell

gcloud notebooks runtimes migrate RUNTIME_ID \
    --project=PROJECT_ID \
    --location=LOCATION \
    --network=NETWORK \
    --subnet=SUBNET \
    --service-account=SERVICE_ACCOUNT \
    --post-startup-script-option=POST_STARTUP_SCRIPT_OPTION

Windows (PowerShell)

gcloud notebooks runtimes migrate RUNTIME_ID `
    --project=PROJECT_ID `
    --location=LOCATION `
    --network=NETWORK `
    --subnet=SUBNET `
    --service-account=SERVICE_ACCOUNT `
    --post-startup-script-option=POST_STARTUP_SCRIPT_OPTION

Windows (cmd.exe)

gcloud notebooks runtimes migrate RUNTIME_ID ^
    --project=PROJECT_ID ^
    --location=LOCATION ^
    --network=NETWORK ^
    --subnet=SUBNET ^
    --service-account=SERVICE_ACCOUNT ^
    --post-startup-script-option=POST_STARTUP_SCRIPT_OPTION

REST

Before using any of the request data, make the following replacements:

PROJECT_ID: Your project ID
LOCATION: The region where your managed notebooks instance is located
RUNTIME_ID: The ID of the managed notebooks instance
NETWORK: Optional: The network that you want to migrate the instance to
SUBNET: Optional: The subnet that you want to migrate the instance to
SERVICE_ACCOUNT: Optional: The email address of the service account that you want to use
POST_STARTUP_SCRIPT_OPTION: Optional: One of the post-startup script options

HTTP method and URL:

POST https://notebooks.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/runtimes/RUNTIME_ID:migrate

Request JSON body:

{
  "network": NETWORK,
  "subnet": SUBNET,
  "serviceAccount": SERVICE_ACCOUNT_EMAIL_ADDRESS,
  "postStartupScriptOption": (POST_STARTUP_SCRIPT_OPTION)
}

To send your request, choose one of these options:

curl

Save the request body in a file named request.json, and execute the following command:

curl -X POST \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     -H "Content-Type: application/json; charset=utf-8" \
     -d @request.json \
     "https://notebooks.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/runtimes/RUNTIME_ID:migrate"

PowerShell

Save the request body in a file named request.json, and execute the following command:

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method POST `
    -Headers $headers `
    -ContentType: "application/json; charset=utf-8" `
    -InFile request.json `
    -Uri "https://notebooks.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/runtimes/RUNTIME_ID:migrate" | Select-Object -Expand Content

Migrate manually

To migrate your instance to a Vertex AI Workbench instance manually, consider using the following methods:

Use Cloud Storage and the terminal: Copy your data and files to Cloud Storage and then to another instance by using the terminal.
Use GitHub: Copy your data and files to a GitHub repository by using the Git extension for JupyterLab.

This guide describes how to migrate data and files by using Cloud Storage and the terminal.

Requirements

You must have terminal access to your managed notebooks instance. Terminal access is manually set when you create an instance. The terminal access setting cannot be changed after the instance is created.

Migrate manually by using Cloud Storage and the terminal

To migrate data and files to a new Vertex AI Workbench instance by using Cloud Storage and the terminal, do the following.

Create a Cloud Storage bucket in the same project where your managed notebooks instance is located.
In that same project, Create a Vertex AI Workbench instance to migrate your data to. When you create this instance:
- Enable terminal access.
- Specify the machine type, network, and other characteristics to match what you need.
In your managed notebooks instance's JupyterLab interface, select File > New > Terminal to open a terminal window.
Use the gsutil tool to copy your user data to a Cloud Storage bucket. The following example command copies all of the files from your instance's /home/jupyter/ directory to a directory in a Cloud Storage bucket.
```
gsutil cp -R /home/jupyter/* gs://BUCKET_NAMEPATH
```
Replace the following:
- BUCKET_NAME: The name of your Cloud Storage bucket
- PATH: The path to the directory where you want to copy your files, for example: /copy/jupyter/
In your new Vertex AI Workbench instance's JupyterLab interface, select File > New > Terminal to open a terminal window.
Use the gsutil tool to copy your data to the new instance. The following example command copies all of the files from a Cloud Storage directory to the your new instance's /home/jupyter/ directory.
```
gsutil cp gs://BUCKET_NAMEPATH* /home/jupyter/
```

Confirm the migration

After the migration, the original managed notebooks instance continues to work as before. Confirm that your migration was a success before you delete the original instance.

Delete the managed notebooks instance

If you don't need the managed notebooks instance that you migrated from, delete it to avoid further charges for that instance.

In the Google Cloud console, go to the Managed notebooks page.

Go to Managed notebooks
Select the instance that you want to delete.
Click Delete. (Depending on the size of your window, the Delete button might be in the options menu.)
To confirm, click Delete.

Troubleshoot

To find methods for diagnosing and resolving migration issues, see Troubleshooting Vertex AI Workbench.

What's next

Learn more about Vertex AI Workbench instances.