Use a post-startup script

This guide describes how to use a post-startup script with Colab Enterprise. This guide also describes how to set environment variables for use with your post-startup script.

Overview

You can use a post-startup script to perform tasks after the startup process of your Colab Enterprise runtime. For example, you can use a post-startup script to install specific packages or make specific changes to your runtime's VM.

To use a post-startup script in Colab Enterprise, you specify the post-startup script when you create a runtime template. Then, when you create a runtime based on that runtime template, the post-startup script runs at the end of the startup process.

Post-startup script location

Your post-startup script URI can be any of the following:

A Cloud Storage URI: For example, gs://BUCKET_NAME/SCRIPT_FILE_NAME. If your post-startup script is stored in Cloud Storage, you must use a runtime with end-user credentials enabled. This is because Colab Enterprise uses your user credentials to access the Cloud Storage bucket.
An HTTPS URL: For example, https://example.com/FOLDER_NAME/SCRIPT_FILE_NAME. Using an HTTPS URL isn't supported when creating your runtime template by using the Google Cloud console.

Post-startup script behavior

You can specify the behavior of your post-startup script. See the following table:

Behavior	Description
Run once (`RUN_ONCE`)	Default. Colab Enterprise runs the post-startup script only once, during runtime creation.
Run every start (`RUN_EVERY_START`)	After every start, Colab Enterprise runs the post-startup script. For example, if idle shutdown stops the runtime, Colab Enterprise runs the post-startup script again when the runtime is restarted.
Download and run every start (`DOWNLOAD_AND_RUN_EVERY_START`)	After every start, Colab Enterprise re-downloads the post-startup script from its source, and then runs the script.

Environment variables

You can specify environment variables in your runtime template, and refer to those variables in your post-startup script.

For example, you can set the quota project using an environment variable. You might set your quota project to a specific project ID by using GOOGLE_CLOUD_QUOTA_PROJECT, a common variable used in Google Cloud client libraries.

To use a post-startup script with environment variables, see Create a runtime template with a post-startup script and environment variables.

Track changes to and recover post-startup scripts

To track changes to your post-startup script and enable recovery of a specific version of the post-startup script, store your post-startup script in a Cloud Storage bucket with Object Versioning enabled. Object Versioning preserves deleted objects as versioned, noncurrent objects that remain accessible in your bucket until explicitly removed.

To specify a version of your post-startup script, add #GENERATION_NUMBER to your post-startup script's URI when you create your runtime template, for example, gs://BUCKET_NAME/FOLDER_NAME/FILE_NAME.py#GENERATION_NUMBER. For more information about Object Versioning and generation numbers, see Object Versioning.

When you use the Google Cloud console to create your runtime template, specifying a generation number for your post-startup script isn't supported.

Before you begin

Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

Roles required to select or create a project

Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
Create a project: To create a project, you need the Project Creator (roles/resourcemanager.projectCreator), which contains the resourcemanager.projects.create permission. Learn how to grant roles.

Go to project selector

Verify that billing is enabled for your Google Cloud project.

Enable the Vertex AI, Dataform, and Compute Engine APIs.

Roles required to enable APIs

To enable APIs, you need the Service Usage Admin IAM role (roles/serviceusage.serviceUsageAdmin), which contains the serviceusage.services.enable permission. Learn how to grant roles.

Enable the APIs

In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

Roles required to select or create a project

Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
Create a project: To create a project, you need the Project Creator (roles/resourcemanager.projectCreator), which contains the resourcemanager.projects.create permission. Learn how to grant roles.

Go to project selector

Verify that billing is enabled for your Google Cloud project.

Enable the Vertex AI, Dataform, and Compute Engine APIs.

Roles required to enable APIs

To enable APIs, you need the Service Usage Admin IAM role (roles/serviceusage.serviceUsageAdmin), which contains the serviceusage.services.enable permission. Learn how to grant roles.

Enable the APIs

Required roles

To get the permissions that you need to create a runtime template in Colab Enterprise, ask your administrator to grant you the Colab Enterprise Admin (roles/aiplatform.colabEnterpriseAdmin) IAM role on the project. For more information about granting roles, see Manage access to projects, folders, and organizations.

You might also be able to get the required permissions through custom roles or other predefined roles.

One or more of the required roles includes the dataform.repositories.list permission. Users who are granted the dataform.repositories.list permission or the Code Creator (roles/dataform.codeCreator) role in a project can list the names of code assets in that project by using the Dataform API or the Dataform command-line interface (CLI). Non-administrators using BigQuery Studio can only see code assets that they created or that were shared with them.

Create a runtime template that uses a post-startup script

To create a runtime template that uses a post-startup script, you can use the Google Cloud console, the Google Cloud CLI, or the REST API.

Console

To create a runtime template that uses a post-startup script, do the following:

In the Google Cloud console, go to the Colab Enterprise Runtime templates page.

Go to Runtime templates
Click New template.

The Create new runtime template dialog appears.
In the Runtime basics section, enter a Display name.
In the Region menu, select the region where you want your runtime template.
In the Environment section, in the Post-startup script field, click Browse, and then navigate to a post-startup script located in Cloud Storage. For more information, see Post-startup script location.
Under the post-startup script URL, select your post-startup script's behavior. The default behavior is Run once. For more information, see Post-startup script behavior.
In the Networking and security section, make sure that Enable end-user credentials is selected.
Complete the rest of the dialog, and then click Create.

Your runtime template appears in the list on the Runtime templates tab.

gcloud

Before using any of the command data below, make the following replacements:

DISPLAY_NAME: the display name of your runtime template.
PROJECT_ID: your project ID.
REGION: the region where you want your runtime template.
POST_STARTUP_SCRIPT_URI: the URI for your post-startup script. For more information, see Post-startup script location.
POST_STARTUP_SCRIPT_BEHAVIOR: the behavior of the post-startup script. See Post-startup script behavior.

Execute the following command:

Linux, macOS, or Cloud Shell

gcloud colab runtime-templates create --display-name="DISPLAY_NAME" \
    --project=PROJECT_ID \
    --region=REGION \
    --machine-type=e2-standard-4 \
    --post-startup-script-url=POST_STARTUP_SCRIPT_URI \
    --post-startup-script-behavior=POST_STARTUP_SCRIPT_BEHAVIOR

Windows (PowerShell)

gcloud colab runtime-templates create --display-name="DISPLAY_NAME" `
    --project=PROJECT_ID `
    --region=REGION `
    --machine-type=e2-standard-4 `
    --post-startup-script-url=POST_STARTUP_SCRIPT_URI `
    --post-startup-script-behavior=POST_STARTUP_SCRIPT_BEHAVIOR

Windows (cmd.exe)

gcloud colab runtime-templates create --display-name="DISPLAY_NAME" ^
    --project=PROJECT_ID ^
    --region=REGION ^
    --machine-type=e2-standard-4 ^
    --post-startup-script-url=POST_STARTUP_SCRIPT_URI ^
    --post-startup-script-behavior=POST_STARTUP_SCRIPT_BEHAVIOR

For more information about the command for creating a runtime template from the command line, see the gcloud CLI documentation.

REST

Before using any of the request data, make the following replacements:

REGION: the region where you want your runtime template.
PROJECT_ID: your project ID.
DISPLAY_NAME: the display name of your runtime template.
POST_STARTUP_SCRIPT_URI: the URI for your post-startup script. For more information, see Post-startup script location.
POST_STARTUP_SCRIPT_BEHAVIOR: the behavior of the post-startup script. See Post-startup script behavior.

HTTP method and URL:

POST https://REGION-aiplatform.googleapis.com/v1beta1/projects/PROJECT_ID/locations/REGION/notebookRuntimeTemplates

Request JSON body:

{
  "displayName": "DISPLAY_NAME",
  "machineSpec": {
    "machineType": "e2-standard-4"
  },
  softwareConfig: {
    postStartupScriptConfig: {
      postStartupScriptUrl: "POST_STARTUP_SCRIPT_URI",
      postStartupScriptBehavior: "POST_STARTUP_SCRIPT_BEHAVIOR"
    },
  },
  "networkSpec": {
    "enableInternetAccess": true
  }
}

To send your request, choose one of these options:

curl

Note: The following command assumes that you have logged in to the gcloud CLI with your user account by running gcloud init or gcloud auth login , or by using Cloud Shell, which automatically logs you into the gcloud CLI . You can check the currently active account by running gcloud auth list.

Save the request body in a file named request.json, and execute the following command:

curl -X POST \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     -H "Content-Type: application/json; charset=utf-8" \
     -d @request.json \
     "https://REGION-aiplatform.googleapis.com/v1beta1/projects/PROJECT_ID/locations/REGION/notebookRuntimeTemplates"

PowerShell

Note: The following command assumes that you have logged in to the gcloud CLI with your user account by running gcloud init or gcloud auth login . You can check the currently active account by running gcloud auth list.

Save the request body in a file named request.json, and execute the following command:

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method POST `
    -Headers $headers `
    -ContentType: "application/json; charset=utf-8" `
    -InFile request.json `
    -Uri "https://REGION-aiplatform.googleapis.com/v1beta1/projects/PROJECT_ID/locations/REGION/notebookRuntimeTemplates" | Select-Object -Expand Content

If successful, the response body contains an instance of Operation.

For more information, see the notebookRuntimeTemplates.create REST API documentation.

Create the runtime

After you create a runtime template that uses your post-startup script, create a runtime based on that runtime template. The runtime starts, and the post-startup script runs according to the specified behavior. See Create a runtime.

Create a runtime template with a post-startup script and environment variables

To create a runtime template that uses a post-startup script and environment variables, you can use the Google Cloud console, the Google Cloud CLI, or the REST API.

Console

To create a runtime template that uses a post-startup script and environment variables, do the following:

In the Google Cloud console, go to the Colab Enterprise Runtime templates page.

Go to Runtime templates
Click New template.

The Create new runtime template dialog appears.
In the Runtime basics section, enter a Display name.
In the Region menu, select the region where you want your runtime template.
In the Environment section, in the Post-startup script field, click Browse, and then navigate to a post-startup script located in Cloud Storage. For more information, see Post-startup script location.
Under the post-startup script URL, select your post-startup script's behavior. The default behavior is Run once. For more information, see Post-startup script behavior.
To add an environment variable, click Add env variable, and then enter a Key and Value pair. To add more environment variables, repeat this step.
In the Networking and security section, make sure that Enable end-user credentials is selected.
Complete the rest of the dialog, and then click Create.

Your runtime template appears in the list on the Runtime templates tab.

gcloud

Before using any of the command data below, make the following replacements:

DISPLAY_NAME: the display name of your runtime template.
PROJECT_ID: your project ID.
REGION: the region where you want your runtime template.
POST_STARTUP_SCRIPT_URI: the URI for your post-startup script. For more information, see Post-startup script location.
POST_STARTUP_SCRIPT_BEHAVIOR: the behavior of the post-startup script. See Post-startup script behavior.
ENV_VARIABLE_NAME: the name of an environment variable to set for your runtime.
ENV_VARIABLE_VALUE: the value of the environment variable to set for your runtime.

Execute the following command:

Linux, macOS, or Cloud Shell

gcloud colab runtime-templates create --display-name="DISPLAY_NAME" \
    --project=PROJECT_ID \
    --region=REGION \
    --machine-type=e2-standard-4 \
    --post-startup-script-url=POST_STARTUP_SCRIPT_URI \
    --post-startup-script-behavior=POST_STARTUP_SCRIPT_BEHAVIOR \
    --set-env-vars=ENV_VARIABLE_NAME=ENV_VARIABLE_VALUE

Windows (PowerShell)

gcloud colab runtime-templates create --display-name="DISPLAY_NAME" `
    --project=PROJECT_ID `
    --region=REGION `
    --machine-type=e2-standard-4 `
    --post-startup-script-url=POST_STARTUP_SCRIPT_URI `
    --post-startup-script-behavior=POST_STARTUP_SCRIPT_BEHAVIOR `
    --set-env-vars=ENV_VARIABLE_NAME=ENV_VARIABLE_VALUE

Windows (cmd.exe)

gcloud colab runtime-templates create --display-name="DISPLAY_NAME" ^
    --project=PROJECT_ID ^
    --region=REGION ^
    --machine-type=e2-standard-4 ^
    --post-startup-script-url=POST_STARTUP_SCRIPT_URI ^
    --post-startup-script-behavior=POST_STARTUP_SCRIPT_BEHAVIOR ^
    --set-env-vars=ENV_VARIABLE_NAME=ENV_VARIABLE_VALUE

For more information about the command for creating a runtime template from the command line, see the gcloud CLI documentation.

REST

Before using any of the request data, make the following replacements:

REGION: the region where you want your runtime template.
PROJECT_ID: your project ID.
DISPLAY_NAME: the display name of your runtime template.
POST_STARTUP_SCRIPT_URI: the URI for your post-startup script. For more information, see Post-startup script location.
POST_STARTUP_SCRIPT_BEHAVIOR: the behavior of the post-startup script. See Post-startup script behavior.
ENV_VARIABLE_NAME: the name of an environment variable to set for your runtime.
ENV_VARIABLE_VALUE: the value of the environment variable to set for your runtime.

HTTP method and URL:

POST https://REGION-aiplatform.googleapis.com/v1beta1/projects/PROJECT_ID/locations/REGION/notebookRuntimeTemplates

Request JSON body:

{
  "displayName": "DISPLAY_NAME",
  "machineSpec": {
    "machineType": "e2-standard-4"
  },
  softwareConfig: {
    postStartupScriptConfig: {
      postStartupScriptUrl: "POST_STARTUP_SCRIPT_URI",
      postStartupScriptBehavior: "POST_STARTUP_SCRIPT_BEHAVIOR"
    },
    "env": [
      {
        "name": "ENV_VARIABLE_NAME",
        "value": "ENV_VARIABLE_VALUE"
      }
    ]
  },
  "networkSpec": {
    "enableInternetAccess": true
  }
}

To send your request, choose one of these options:

curl

Save the request body in a file named request.json, and execute the following command:

curl -X POST \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     -H "Content-Type: application/json; charset=utf-8" \
     -d @request.json \
     "https://REGION-aiplatform.googleapis.com/v1beta1/projects/PROJECT_ID/locations/REGION/notebookRuntimeTemplates"

PowerShell

Save the request body in a file named request.json, and execute the following command:

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method POST `
    -Headers $headers `
    -ContentType: "application/json; charset=utf-8" `
    -InFile request.json `
    -Uri "https://REGION-aiplatform.googleapis.com/v1beta1/projects/PROJECT_ID/locations/REGION/notebookRuntimeTemplates" | Select-Object -Expand Content

If successful, the response body contains an instance of Operation.

For more information, see the notebookRuntimeTemplates.create REST API documentation.

Create the runtime

Limitations

Consider the following limitations when using post-startup scripts with Colab Enterprise:

To create a runtime based on a runtime template that includes a post-startup script, you must be the owner and creator of that runtime template.
Passing environment variables through the post-startup script isn't supported. You must specify environment variables in the software configuration as described in this guide.
To use a post-startup script located in Cloud Storage, you must use a runtime with end-user credentials enabled and your user credentials must have access to the Cloud Storage bucket.
If you use the Google Cloud console to create your runtime template, your post-startup script must be located in Cloud Storage. Post-startup scripts in other locations aren't supported when creating your runtime template by using the Google Cloud console.
Specifying a generation number for your post-startup script isn't supported when you use the Google Cloud console to create your runtime template.

What's next

Create a runtime and connect to it.
To manage your runtime, see Manage runtimes.
Learn more about runtimes and runtime templates.