Set up Datalab to use with Cloud Monitoring
This Quickstart shows you how to set up Datalab to use with your Cloud Monitoring projects. Datalab's dynamic notebooks let you perform ad hoc analyses and visualizations that go beyond the present features of Monitoring.
To preview the Monitoring tutorials in Datalab, see Monitoring tutorials. You cannot interact with the tutorials unless you are running Datalab.
Before you begin
- Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.
-
In the Google Cloud console, on the project selector page, select or create a Google Cloud project.
-
Make sure that billing is enabled for your Cloud project. Learn how to check if billing is enabled on a project.
-
Enable the Compute Engine and Cloud Source Repositories APIs.
- Install the Google Cloud CLI.
-
To initialize the gcloud CLI, run the following command:
gcloud init
-
In the Google Cloud console, on the project selector page, select or create a Google Cloud project.
-
Make sure that billing is enabled for your Cloud project. Learn how to check if billing is enabled on a project.
-
Enable the Compute Engine and Cloud Source Repositories APIs.
- Install the Google Cloud CLI.
-
To initialize the gcloud CLI, run the following command:
gcloud init
Set up Datalab
Perform the following steps from a terminal window on your local machine:
- Get the latest
gcloud
command:gcloud components update
- Install the
gcloud datalab
component:gcloud components install datalab
To configure
gcloud
to connect to your Google Cloud project ID, run the following command after replacing[PROJECT_ID]
with your Google Cloud project ID:gcloud config set project [PROJECT_ID]
To verify the configuration, run the following command:
gcloud config get-value project
To create a Datalab instance, run the following command after replacing
[DATALAB-INSTANCE-NAME]
with the name of your instance. Names must begin with a lowercase letter, be followed by no more than 62 lowercase letters, numbers, or hyphens, and must not end with a hyphen:datalab create [DATALAB-INSTANCE-NAME]
- If prompted, select a zone from the list.
- If you are asked to set a SSH passphrase, enter a phrase. Be sure to save this passphrase.
After a few moments, a message similar to the one below is displayed.
Created [https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/zones/[ZONE]/instances/[DATALAB-INSTANCE-NAME]]. Connecting to [DATALAB-INSTANCE-NAME]. This will create an SSH tunnel and may prompt you to create an rsa key pair. To manage these keys, see https://cloud.google.com/compute/docs/instances/adding-removing-ssh-keys Waiting for Datalab to be reachable at http://localhost:8081/
If prompted, enter your passphrase:
Enter passphrase for key '[...]/.ssh/google_compute_engine':
After a few minutes, the create command completes successfully and a message similar to the one below is displayed:
The connection to Datalab is now open and will remain until this command is killed. You can connect to Datalab at http://localhost:8081/
The connection to your Datalab instance remains open while the
datalab
command is active. If the terminal command window is closed or interrupted, the connection is terminated. To re-establish the connection, run the following command:datalab connect [DATALAB-INSTANCE-NAME]
The Datalab Interface
When you execute the datalab connect
command, a browser window is opened
with the URL set to http://127.0.0.1:8081/
. This page displays the
Datalab home page:
In the Datalab docs/ folder there are several Datalab getting started notebooks you can explore, including notebooks for Cloud Monitoring.
Monitoring tutorials
Datalab is installed with several Monitoring interactive tutorials:
- Getting started
- Shows how to import the Python Google Cloud's operations suite API into Datalab and set your default Google Cloud project ID. There is sample code that calls the API and retrieves monitoring data from your project.
- Group metrics
- Shows how to look at the group structure in a project and how to use groups to filter and aggregate metric data. To use group metrics, you must configure a Cloud Monitoring group. For details more details, see Using resource groups.
- Time-shifted data
- Shows how to transform time series data in interesting ways. Since your project might not have enough VM instances to be a good example, the tutorial is set up to optionally use previously-extracted data from a demonstration project.
You can run the tutorials, and optionally modify them, as explained in the following section.
Running the tutorials
To run the tutorials, do the following:
Click on docs, tutorials, and Stackdriver Monitoring in the Datalab interface. You should see the following page:
Select the tutorial. Click on the name of the tutorial you want to run.
Set the project ID: Edit the cell containing
set_datalab_project_id('my-project-id')
and replacemy-project-id
with your Google Cloud project ID.Run the tutorial code. From the menu bar at the top of the interface, select Run > Run all cells. This reruns all the code in the tutorial using your current project ID.
Feel free to modify the code in the tutorials and try out your changes. You can run the code in a single cell by clicking on the cell and choosing Run from the menu to the left of the cell. You can also create your own notebooks.
Clean up
To avoid incurring charges to your Google Cloud account for the resources used on this page, follow these steps.
Click on the Running Sessions icon at the top-right of the Datalab interface, and shut down any notebooks you are not using. You can restart them later if you need to. Close the associated browser tabs or windows.
Enter
CTRL-C
in the window where Datalab is running and close the Datalab tab in your browser.You incur charges from the time of creation to the time of deletion of the Datalab VM instance (see Cloud Datalab Pricing). You are also charged for the Persistent Disk where notebooks are stored. The Persistent Disk remains after the deletion of the Datalab VM until you delete it.
To delete the Datalab VM instance and its Persistent Disk, run the following:
datalab delete --delete-disk [DATALAB-INSTANCE-NAME]
There are additional resources are created by the
datalab create
command that are reused by other Datalab instances that you create. You can run the following commands to delete the additional resources listed below if you don't expect to create additional Datalab instances:- Delete the
datalab-network-allow-ssh
firewall rule, which allows SSH connections to your Datalab instances:gcloud compute firewall-rules delete datalab-network-allow-ssh
- Delete the
datalab-network
Virtual Private Cloud (VPC) network, to which Datalab instances are connected by default:gcloud compute networks delete datalab-network
- Delete the
datalab-notebooks
Cloud Source Repository, which is set up for you to store your notebooks (see Working with notebooks if you wish to backup notebooks before deleting the repo):gcloud source repos delete datalab-notebooks
- Delete the
If you created any VM instances or projects that you don't want to keep, remove them.
What's next
See the following API reference material:
Learn more about DevOps and explore DORA's research program.