Schedule a notebook run

This page shows you how to schedule a notebook run in Colab Enterprise.

Overview

You can schedule a notebook to run immediately one time, or on a recurring schedule.

When you schedule the notebook run, you select a runtime template. Colab Enterprise uses this runtime template to create the runtime that runs your notebook.

The runtime needs specific permissions to run the notebook's code and access Google Cloud services and APIs.

  • If your runtime template configuration has end-user credentials enabled, then the runtime uses the permissions associated with your user credentials.

  • If end-user credentials aren't enabled, you must specify a service account when you schedule the notebook run. Colab Enterprise uses this service account's credentials to run your notebook.

For more information, see Required roles for running the notebook.

After Colab Enterprise completes the notebook run, the results are stored in a shareable Cloud Storage bucket.

Limitations

Colab Enterprise runtimes use Compute Engine quota. See the Compute Engine Allocation quotas page.

Before you begin

  1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.
  2. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

    Go to project selector

  3. Make sure that billing is enabled for your Google Cloud project.

  4. Enable the Vertex AI, Dataform, and Compute Engine APIs.

    Enable the APIs

  5. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

    Go to project selector

  6. Make sure that billing is enabled for your Google Cloud project.

  7. Enable the Vertex AI, Dataform, and Compute Engine APIs.

    Enable the APIs

Required roles for scheduling the notebook run

To ensure that your user account has the necessary permissions to schedule a notebook run in Colab Enterprise, ask your administrator to grant your user account the following IAM roles on the project:

For more information about granting roles, see Manage access to projects, folders, and organizations.

Your administrator might also be able to give your user account the required permissions through custom roles or other predefined roles.

Required roles for running the notebook

The principal that runs the notebook needs specific permissions. The principal is either your user account or a service account that you specify, as described in the overview.

To ensure that the principal has the necessary permissions to run a notebook in Colab Enterprise, ask your administrator to grant the principal the following IAM roles:

For more information about granting roles, see Manage access to projects, folders, and organizations.

These predefined roles contain the permissions required to run a notebook in Colab Enterprise. To see the exact permissions that are required, expand the Required permissions section:

Required permissions

The following permissions are required to run a notebook in Colab Enterprise:

  • dataform.locations.list on the notebook
  • dataform.repositories.computeAccessTokenStatus on the notebook
  • dataform.repositories.fetchHistory on the notebook
  • dataform.repositories.fetchRemoteBranches on the notebook
  • dataform.repositories.get on the notebook
  • dataform.repositories.getIamPolicy on the notebook
  • dataform.repositories.list on the notebook
  • dataform.repositories.queryDirectoryContents on the notebook
  • dataform.repositories.readFile on the notebook
  • logging.logEntries.create on the project
  • logging.logEntries.route on the project
  • monitoring.metricDescriptors.create on the project
  • monitoring.metricDescriptors.get on the project
  • monitoring.metricDescriptors.list on the project
  • monitoring.monitoredResourceDescriptors.get on the project
  • monitoring.monitoredResourceDescriptors.list on the project
  • monitoring.timeSeries.create on the project
  • resourcemanager.projects.get on the project
  • resourcemanager.projects.list on the project
  • storage.buckets.get on the notebook
  • storage.managedFolders.create on the notebook
  • storage.managedFolders.delete on the notebook
  • storage.managedFolders.get on the notebook
  • storage.managedFolders.list on the notebook
  • storage.multipartUploads.abort on the notebook
  • storage.multipartUploads.create on the notebook
  • storage.multipartUploads.list on the notebook
  • storage.multipartUploads.listParts on the notebook
  • storage.objects.create on the notebook
  • storage.objects.delete on the notebook
  • storage.objects.get on the notebook
  • storage.objects.list on the notebook
  • storage.objects.restore on the notebook
  • storage.objects.setRetention on the notebook

Your administrator might also be able to give the principal these permissions with custom roles or other predefined roles.

Run a notebook once

You can run a notebook one time by using the Google Cloud console or the Vertex AI Python client library.

Console

  1. In the Google Cloud console, go to the Colab Enterprise Notebooks page.

    Go to Notebooks

  2. In the Region menu, select the region that contains your notebook.

  3. Next to a notebook, click the Notebook actions menu and select Schedule.

  4. In the Schedule name field, enter a name for your schedule.

  5. Click the Runtime template list, and select a runtime template. The runtime template determines the specifications of the runtime that runs your notebook.

  6. Under Run schedule, select One-off to run your notebook as soon as you submit the notebook run.

  7. Next to the Cloud Storage output location field, click Browse to open the Select folder dialog.

  8. Select a Cloud Storage bucket. Or, to create a bucket, click  Create new bucket and complete the dialog.

  9. If you selected a runtime template without end-user credentials enabled, the dialog includes a Service account field. In the Service account field, enter a service account's email address.

  10. Click Submit.

    The notebook run starts immediately.

Python

Before trying this sample, install the Vertex AI SDK for Python. The Vertex AI Python client library is installed when you install the Vertex AI SDK for Python. For more information, see the Vertex AI SDK for Python API reference documentation.

from google.cloud import aiplatform_v1beta1

PROJECT_ID = "my-project"
LOCATION = "us-central1"

API_ENDPOINT = f"{LOCATION}-aiplatform.googleapis.com"
PARENT = f"projects/{PROJECT_ID}/locations/{LOCATION}"

notebook_service_client = aiplatform_v1beta1.NotebookServiceClient(client_options = {
    "api_endpoint": API_ENDPOINT,
})

operation = notebook_service_client.create_notebook_execution_job(parent=PARENT, notebook_execution_job={
    "display_name": "my-execution-job",

    # Specify a NotebookRuntimeTemplate to source compute configuration from
    "notebook_runtime_template_resource_name": f"projects/{PROJECT_ID}/locations/{LOCATION}/notebookRuntimeTemplates/{template_id}",

    # Specify a Colab Enterprise notebook to execute
    "dataformRepositorySource": {
        "dataformRepositoryResourceName": f"projects/{PROJECT_ID}/locations/{LOCATION}/repositories/{repository_id}",
    },

    # Specify a Cloud Storage bucket to store output artifacts
    "gcs_output_uri": "gs://my-bucket/,

    # Specify the identity to run the execution as
    "execution_user": {EMAIL},

    # Run as the service account instead
    # "service_account": "my-service-account",
})
print("Waiting for operation to complete...")
result = operation.result()

You can view results from completed notebook runs on the Execution jobs tab.

Schedule a notebook run

You can schedule a notebook run by using the Google Cloud console or the Vertex AI Python client library.

Console

  1. In the Google Cloud console, go to the Colab Enterprise Notebooks page.

    Go to Notebooks

  2. In the Region menu, select the region that contains your notebook.

  3. Next to a notebook, click the Notebook actions menu and select Schedule.

  4. In the Schedule name field, enter a name for your schedule.

  5. Click the Runtime template list, and select a runtime template. The runtime template determines the specifications of the runtime that runs your notebook.

  6. Under Run schedule, select Recurring to schedule the notebook run for a specific interval of time.

  7. Complete the scheduling dialog.

  8. Next to the Cloud Storage output location field, click Browse to open the Select folder dialog.

  9. Select a Cloud Storage bucket. Or, to create a bucket, click  Create new bucket and complete the dialog.

  10. If you selected a runtime template without end-user credentials enabled, the dialog includes a Service account field. In the Service account field, enter a service account's email address.

  11. Click Submit.

    Scheduled notebook runs start automatically on the schedule that you set.

Python

Before trying this sample, install the Vertex AI SDK for Python. The Vertex AI Python client library is installed when you install the Vertex AI SDK for Python. For more information, see the Vertex AI SDK for Python API reference documentation.

from google.cloud import aiplatform_v1beta1

PROJECT_ID = "my-project"
LOCATION = "us-central1"

API_ENDPOINT = f"{LOCATION}-aiplatform.googleapis.com"
PARENT = f"projects/{PROJECT_ID}/locations/{LOCATION}"

schedules_service_client = aiplatform_v1beta1.ScheduleServiceClient(client_options = {
    "api_endpoint": API_ENDPOINT,
})

schedule = schedules_service_client.create_schedule(parent=PARENT, schedule={
    "display_name": "my-notebook-schedule",

    # Time specification. TZ is optional.
    # cron = "* * * * *" to execute it in the next minute.
    "cron": "TZ=America/Los_Angeles * * * * *",

    # How many runs the schedule will trigger before it becomes COMPLETED.
    # A Schedule in COMPLETED state will not trigger any more runs.
    "max_run_count": 1,
    "max_concurrent_run_count": 1,

    "create_notebook_execution_job_request": {
      "parent": PARENT,
      "notebook_execution_job": {
        "display_name": "my-execution-job",

        # Specify a NotebookRuntimeTemplate to source compute configuration from
        "notebook_runtime_template_resource_name": f"projects/{PROJECT_ID}/locations/{LOCATION}/notebookRuntimeTemplates/{template_id}",

        # Specify a Colab Enterprise notebook to execute
        "dataformRepositorySource": {
            "dataformRepositoryResourceName": f"projects/{PROJECT_ID}/locations/{LOCATION}/repositories/{repository_id}",
        },

        # Specify a Cloud Storage bucket to store output artifacts
        "gcs_output_uri": "gs://my-bucket/,


        # Specify the identity to run the execution as
        "execution_user": {EMAIL},

        # Run as the service account instead
        # "service_account": "my-service-account",
    }
  }
})

In the Google Cloud console, you can view your schedules on the Schedules tab. You can view results from the completed notebook runs on the Execution jobs tab.

View results

You can view notebook run results by using the Google Cloud console or the Vertex AI Python client library.

Console

  1. In the Google Cloud console, go to the Colab Enterprise Execution jobs page.

    Go to Execution jobs

  2. Next to the notebook run that you want to view results for, click View result.

    A read-only panel opens that shows the results of the notebook run.

  3. To close the panel, click Close.

Python

Before trying this sample, install the Vertex AI SDK for Python. The Vertex AI Python client library is installed when you install the Vertex AI SDK for Python. For more information, see the Vertex AI SDK for Python API reference documentation.

from google.cloud import aiplatform_v1beta1

PROJECT_ID = "my-project"
LOCATION = "us-central1"

API_ENDPOINT = f"{LOCATION}-aiplatform.googleapis.com"
PARENT = f"projects/{PROJECT_ID}/locations/{LOCATION}"

notebook_service_client = aiplatform_v1beta1.NotebookServiceClient(client_options = {
    "api_endpoint": API_ENDPOINT,
})

notebook_execution_jobs = notebook_service_client.list_notebook_execution_jobs(parent=PARENT)
notebook_execution_jobs

Delete results

To delete a result from one of your notebook runs, do the following:

  1. In the Google Cloud console, go to the Colab Enterprise Execution jobs page.

    Go to Execution jobs

  2. Select the notebook run that you want to delete the result for.

  3. Click  Delete.

  4. To confirm the deletion, click Confirm.

Share a notebook run's results

You can share notebook run results by providing access to the Cloud Storage bucket that contains your notebook run. Providing this access also grants users access to any other resources in the same Cloud Storage bucket (see Security considerations).

For more information, see the Cloud Storage Sharing and collaboration page.

Security considerations

Your notebook run results are stored as notebook (IPYNB) files in a Cloud Storage bucket. Consider the following when you grant access to this bucket:

  • Anyone with access to the bucket can see the notebook file's code and the results of the notebook run.

  • Anyone with the ability to change the contents of the bucket can change the contents of the notebook file.

View schedule details

You can view information about a schedule, including:

  • The Cloud Storage bucket that the schedule stores results in.
  • The start and end time.
  • The frequency.

To view schedule details, do the following:

  1. In the Google Cloud console, go to the Colab Enterprise Schedules page.

    Go to Schedules

  2. Click the name of a schedule.

    The Schedule details page opens.

  3. To go back to the Schedules page, click  Back to previous page.

Pause, resume, or delete a schedule

To pause, resume, or delete a schedule, do the following:

  1. In the Google Cloud console, go to the Colab Enterprise Schedules page.

    Go to Schedules

  2. Select a schedule.

  3. Click  Pause,  Resume, or  Delete.

What's next