Determining server usage for reporting

Through the Stackdriver logging tool available in the GCP Console, you can view historical server usage data. This can be useful so you can view each time a VM in a specific node is placed on a physical server and get the physical server IDs.

Before you begin

To view physical server usage on Stackdriver Logging, you must first enable the Stackdriver Logging API before launching VMs onto sole-tenant nodes so you can collect data from the beginning. If you have not enabled logging and have already launched VMs, you can still enable it the service but you will only be able to access data gathered from when logging was enabled.

Enable the Stackdriver Logging API

Getting server IDs

Permissions required for this task

To perform this task, you must have the following permissions.

• logging.logEntries.list on the project
• logging.logMetrics.list on the project
• logging.logMetrics.get on the project
• logging.logs.list on the project
• logging.logServiceIndexes.list on the project
• logging.logServices.list on the project
• logging.sinks.list on the project
• logging.sinks.get on the project
• logging.usage.get on the project
• resourcemanager.projects.get on the project

To get the physical server ID of the your sole-tenant nodes:

1. Go to the Stackdriver Logging > Logs (Logs Viewer) page in the GCP Console:

   Go to the Logs Viewer page

2. Select an existing GCP project if prompted.

3. In the first drop-down menu, select GCE VM Instance.

4. In the third drop-down menu, select the time limit to restrict results.

5. In the text box, enter text:NotifyInstanceLocation.

   

6. Expand the results of each entry and look for the serverId. The serverId is a unique identifier for the physical machine backing your sole-tenant nodes. Physical servers backing the n1-node-96-624 node-type possess 56 physical cores.

   

Note: For certain types of events, entries may be duplicated. Duplicate entries can be identified using the timestamp field associated with the instance location in the log and deduplicated accordingly.

Exporting license data to BigQuery

Permissions required for this task

To perform this task, you must have the following permissions.

• All the permissions to view logs
• logging.sinks.create on the project
• logging.sinks.update on the project
• logging.sinks.delete on the project

System events information can be exported to a BigQuery dataset in order to determine per-core or per-socket usage:

1. In the first drop-down menu, ensure GCE VM Instance is selected.
2. Click Create Export at the top of the Logs Viewer page.
3. Create a data sink by entering a Sink Name, selecting BigQuery as the Sink Service, and selecting a BigQuery dataset.

Once created, the data sink will receive all subsequent audit logging activity. This means that BigQuery will receive all future-looking data on server usage. Note that any data prior to setting up BigQuery integration will NOT be accessible.

Sample queries to identify server usage in BigQuery

This section assumes you are familiar with running queries in BigQuery.

Identify list of instances using your custom images with the following query on the cloudaudit_googleapis_com_activity_[DATE] table:

-- QUERY TO IDENTIFY ALL INSTANCE IDs USING A SPECIFIC SOURCE IMAGE
SELECT resource.labels.instance_id, resource.labels.project_id, resource.labels.zone, disks.initializeparams.sourceimage
FROM `[MY_BYOL_PROJECT_SINK_NAME].cloudaudit_googleapis_com_activity_*`, UNNEST(protopayload_auditlog.request_instances_insert.disks) AS disks
WHERE disks.initializeparams.sourceimage LIKE '%[BYOL_IMAGE_NAME]%'

Next, using the VM instances, the hosts for each instance can be identified through the following query

-- QUERY TO IDENTIFY ALL SERVER IDs USED BY A PROJECT
SELECT protopayload_auditlog.metadata_audit_gceinstancelocationmetadata.serverid, protopayload_auditlog.metadata_audit_gceinstancelocationmetadata.timestamp,
56 AS physical_cores
FROM `[MY_BYOL_PROJECT_SINK_NAME].cloudaudit_googleapis_com_system_event_*`
GROUP BY 1,2

A joint query example:

-- QUERY TO IDENTIFY ALL INSTANCE IDs USING A SPECIFIC SOURCE IMAGE
SELECT
resource.labels.instance_id, protopayload_auditlog.metadata_audit_gceinstancelocationmetadata.serverid as server_id, protopayload_auditlog.metadata_audit_gceinstancelocationmetadata.timestamp,
56 AS physical_cores,
2 AS physical_sockets
FROM `[MY_BYOL_PROJECT_SINK_NAME].cloudaudit_googleapis_com_system_event_*`
WHERE resource.labels.instance_id IN (
SELECT resource.labels.instance_id
FROM `[MY_BYOL_PROJECT_SINK_NAME].cloudaudit_googleapis_com_activity_*`, UNNEST(protopayload_auditlog.request_instances_insert.disks) AS disks
WHERE disks.initializeparams.sourceimage LIKE '%[BYOL_IMAGE_NAME]%')
GROUP BY 1,2,3
ORDER BY 1 ASC,3 ASC;

where:

• [MY_BYOL_PROJECT_SINK_NAME] is the name of the data sink you created.
• [BYOL_IMAGE_NAME] is the name of the image used for bring your own license.

For information on using wildcards for selecting dates and other helpful hints for querying cloud audit logs on BigQuery, please view cloud audit logs documentation.

What's next

• Review how to Bring your own license using sole-tenant nodes.
• Learn more about Sole-tenant nodes.
• See the Frequently asked questions for bringing your own license.
• Review documentation for Creating and starting an instance.
• Review the Images documentation.