Cloud Audit Logs Tutorial

---

Note: This content applies only to Cloud Run functions—formerly Cloud Functions (2nd gen).

This tutorial demonstrates writing, deploying, and triggering an event-driven Cloud Run function with a Cloud Audit Logs trigger.

Cloud Run functions enables your functions to be triggered by Cloud Audit Logs entries. Many Google Cloud products write to Cloud Audit Logs when important in-product actions occur. These log entries can trigger the execution of Cloud Run functions in real time, which allows users to automatically process and/or act on them.

These logs are generated by many different events in Google Cloud and cover most Google Cloud products. Thus, Cloud Audit Logs triggers enable you to create functions that react to most state changes in Google Cloud.

This tutorial will show you how use Cloud Audit Logs triggers to label newly created Compute Engine instances with the name of the entity (person or service account) that created them.

If you are new to Cloud Audit Logs and want to learn more, see the Cloud Audit Logs documentation.

Objectives

• Write an event-driven Cloud Run function that receives a Cloud Audit Logs event when a Compute Engine VM instance is created.
• Trigger the function by creating a Compute Engine VM instance, at which point the instance will be labeled with the name of the entity (person or service account) that created it.

Costs

In this document, you use the following billable components of Google Cloud:

• Cloud Run functions
• Cloud Build
• Pub/Sub
• Artifact Registry
• Eventarc
• Cloud Logging
• Compute Engine

For details, see Cloud Run functions pricing.

To generate a cost estimate based on your projected usage, use the pricing calculator. New Google Cloud users might be eligible for a free trial.

Before you begin

1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

2. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   Go to project selector

3. Make sure that billing is enabled for your Google Cloud project.

4. Enable the Cloud Functions, Cloud Run, Cloud Build, Artifact Registry, Eventarc, Logging, Compute Engine, and Pub/Sub APIs.

   Enable the APIs

5. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   Go to project selector

6. Make sure that billing is enabled for your Google Cloud project.

7. Enable the Cloud Functions, Cloud Run, Cloud Build, Artifact Registry, Eventarc, Logging, Compute Engine, and Pub/Sub APIs.

   Enable the APIs

8. Install and initialize the Cloud SDK.
9. Update gcloud components:

gcloud components update

Need a command prompt? You can use the Google Cloud Shell. The Google Cloud Shell is a command line environment that already includes the Google Cloud SDK, so you don't need to install it. The Google Cloud SDK also comes preinstalled on Compute Engine Virtual Machines.

10. Prepare your development environment.

    Node.js

    Go to the Node.js setup guide

    Python

    Go to the Python setup guide

    Go

    Go to the Go setup guide

    Java

    Go to the Java setup guide

Prerequisites

1. Open the IAM & Admin > Audit Logs page in the Google Cloud console:

   Go to the IAM & Admin > Audit Logs page

2. Enable the Cloud Audit Logs Admin Read, Data Read, and Data Write Log Types for the Compute Engine API:

   

3. Check whether the Compute Engine Service Account has the Editor role. This service account will be used as the service identity for Cloud Run functions.

   Go to the IAM & Admin > IAM page

   Find the entry [e-mail] in the table and look at the Roles column. If the column contains Editor you can skip the following steps. Otherwise, go to the next steps and assign the necessary roles to the service account.

4. Grant the eventarc.eventReceiver role to the project's Compute Engine service account:

   PROJECT_ID=$(gcloud config get-value project)
   PROJECT_NUMBER=$(gcloud projects list --filter="project_id:$PROJECT_ID" --format='value(project_number)')
   
   # Allow service account token creation
   gcloud projects add-iam-policy-binding $PROJECT_ID \
    --member serviceAccount:$[e-mail] \
    --role roles/eventarc.eventReceiver
   

5. Grant the run.invoker role to the project's Compute Engine service account so that the Pub/Sub trigger can execute the function:

   PROJECT_ID=$(gcloud config get-value project)
   PROJECT_NUMBER=$(gcloud projects list --filter="project_id:$PROJECT_ID" --format='value(project_number)')
   
   # Allow service account token creation
   gcloud projects add-iam-policy-binding $PROJECT_ID \
    --member serviceAccount:$[e-mail] \
    --role roles/run.invoker
   

6. Grant the compute.instanceAdmin role to the project's Compute Engine service account so that the function code has the necessary permissions to get VM instances and set labels on them:

   PROJECT_ID=$(gcloud config get-value project)
   PROJECT_NUMBER=$(gcloud projects list --filter="project_id:$PROJECT_ID" --format='value(project_number)')
   
   # Allow service account token creation
   gcloud projects add-iam-policy-binding $PROJECT_ID \
    --member serviceAccount:$[e-mail] \
    --role roles/compute.instanceAdmin
   

Preparing the application

1. Clone the sample app repository to your local machine:

   Node.js

   git clone https://github.com/GoogleCloudPlatform/nodejs-docs-samples.git

   Alternatively, you can download the sample as a zip file and extract it.

   Python

   git clone https://github.com/GoogleCloudPlatform/python-docs-samples.git

   Alternatively, you can download the sample as a zip file and extract it.

   Go

   git clone https://github.com/GoogleCloudPlatform/golang-samples.git

   Alternatively, you can download the sample as a zip file and extract it.

   Java

   git clone https://github.com/GoogleCloudPlatform/java-docs-samples.git

   Alternatively, you can download the sample as a zip file and extract it.

2. Change to the directory that contains the Cloud Run functions sample code for accessing Cloud Audit Logs:

   Node.js

   cd nodejs-docs-samples/functions/v2/autoLabelInstance/

   Python

   cd python-docs-samples/functions/v2/label_gce_instance/

   Go

   cd golang-samples/functions/functionsv2/label_gce_instance/

   Java

   cd java-docs-samples/functions/v2/label-compute-instance/

3. Take a look at the sample code:

   Node.js

   const functions = require('@google-cloud/functions-framework');
   
   const compute = require('@google-cloud/compute');
   const instancesClient = new compute.InstancesClient();
   
   // Register a CloudEvent callback with the Functions Framework that labels
   // newly-created GCE instances with the entity (person or service account)
   // that created them.
   functions.cloudEvent('autoLabelInstance', async cloudEvent => {
     // Extract parameters from the CloudEvent + Cloud Audit Log data
     const payload = cloudEvent.data && cloudEvent.data.protoPayload;
     const authInfo = payload && payload.authenticationInfo;
     let creator = authInfo && authInfo.principalEmail;
   
     // Get relevant VM instance details from the CloudEvent's `subject` property
     // Example value:
     //   compute.googleapis.com/projects/<PROJECT>/zones/<ZONE>/instances/<INSTANCE>
     const params = cloudEvent.subject && cloudEvent.subject.split('/');
   
     // Validate data
     if (!creator || !params || params.length !== 7) {
       throw new Error('Invalid event structure');
     }
   
     // Format the 'creator' parameter to match GCE label validation requirements
     creator = creator.toLowerCase().replace(/\W/g, '_');
   
     // Get the newly-created VM instance's label fingerprint
     // This is required by the Compute Engine API to prevent duplicate labels
     const getInstanceRequest = {
       project: params[2],
       zone: params[4],
       instance: params[6],
     };
     const [instance] = await instancesClient.get(getInstanceRequest);
   
     // Label the instance with its creator
     const setLabelsRequest = Object.assign(
       {
         instancesSetLabelsRequestResource: {
           labels: {creator},
           labelFingerprint: instance.labelFingerprint,
         },
       },
       getInstanceRequest
     );
   
     return instancesClient.setLabels(setLabelsRequest);
   });

   Python

   import re
   
   from google.api_core.exceptions import GoogleAPIError
   from google.cloud import compute_v1
   from google.cloud.compute_v1.types import compute
   
   instances_client = compute_v1.InstancesClient()
   
   
   # CloudEvent function that labels newly-created GCE instances
   # with the entity (user or service account) that created them.
   #
   # @param {object} cloudevent A CloudEvent containing the Cloud Audit Log entry.
   # @param {object} cloudevent.data.protoPayload The Cloud Audit Log entry.
   def label_gce_instance(cloudevent):
       # Extract parameters from the CloudEvent + Cloud Audit Log data
       payload = cloudevent.data.get("protoPayload", dict())
       auth_info = payload.get("authenticationInfo", dict())
       creator = auth_info.get("principalEmail")
   
       # Get relevant VM instance details from the cloudevent's `subject` property
       # Example value:
       #   compute.googleapis.com/projects/<PROJECT_ID>/zones/<ZONE_ID>/instances/<INSTANCE_NAME>
       instance_params = cloudevent["subject"].split("/")
   
       # Validate data
       if not creator or not instance_params or len(instance_params) != 7:
           # This is not something retries will fix, so don't throw an Exception
           # (Thrown exceptions trigger retries *if* you enable retries in GCF.)
           print("ERROR: Invalid `principalEmail` and/or CloudEvent `subject`.")
           return
   
       instance_project = instance_params[2]
       instance_zone = instance_params[4]
       instance_name = instance_params[6]
   
       # Format the 'creator' parameter to match GCE label validation requirements
       creator = re.sub("\\W", "_", creator.lower())
   
       # Get the newly-created VM instance's label fingerprint
       # This is required by the Compute Engine API to prevent duplicate labels
       instance = instances_client.get(
           project=instance_project, zone=instance_zone, instance=instance_name
       )
   
       # Construct API call to label the VM instance with its creator
       request_init = {
           "project": instance_project,
           "zone": instance_zone,
           "instance": instance_name,
       }
       request_init[
           "instances_set_labels_request_resource"
       ] = compute.InstancesSetLabelsRequest(
           label_fingerprint=instance.label_fingerprint, labels={"creator": creator}
       )
       request = compute.SetLabelsInstanceRequest(request_init)
   
       # Perform instance-labeling API call
       try:
           instances_client.set_labels_unary(request)
           print(f"Labelled VM instance {instance_name} with creator: {creator}")
       except GoogleAPIError as e:
           # Swallowing the exception means failed invocations WON'T be retried
           print("Label operation failed", e)
   
           # Uncomment the line below to retry failed invocations.
           # (You'll also have to enable retries in Cloud Functions itself.)
           # raise e
   
       return
   
   

   Go

   
   // Package helloworld provides a set of Cloud Functions samples.
   package helloworld
   
   import (
   	"context"
   	"fmt"
   	"log"
   	"regexp"
   	"strings"
   
   	compute "cloud.google.com/go/compute/apiv1"
   	"cloud.google.com/go/compute/apiv1/computepb"
   	"github.com/GoogleCloudPlatform/functions-framework-go/functions"
   	"github.com/cloudevents/sdk-go/v2/event"
   	"google.golang.org/protobuf/proto"
   )
   
   // AuditLogEntry represents a LogEntry as described at
   // https://cloud.google.com/logging/docs/reference/v2/rest/v2/LogEntry
   type AuditLogEntry struct {
   	ProtoPayload *AuditLogProtoPayload `json:"protoPayload"`
   }
   
   // AuditLogProtoPayload represents AuditLog within the LogEntry.protoPayload
   // See https://cloud.google.com/logging/docs/reference/audit/auditlog/rest/Shared.Types/AuditLog
   type AuditLogProtoPayload struct {
   	MethodName         string                 `json:"methodName"`
   	ResourceName       string                 `json:"resourceName"`
   	AuthenticationInfo map[string]interface{} `json:"authenticationInfo"`
   }
   
   var client *compute.InstancesClient
   
   func init() {
   	// Create an Instances Client
   	var err error
   	client, err = compute.NewInstancesRESTClient(context.Background())
   	if err != nil {
   		log.Fatalf("Failed to create instances client: %s", err)
   	}
   
   	functions.CloudEvent("label-gce-instance", labelGceInstance)
   }
   
   // Cloud Function that receives GCE instance creation Audit Logs, and adds a
   // `creator` label to the instance.
   func labelGceInstance(ctx context.Context, ev event.Event) error {
   	// Extract parameters from the Cloud Event and Cloud Audit Log data
   	logentry := &AuditLogEntry{}
   	if err := ev.DataAs(logentry); err != nil {
   		err = fmt.Errorf("event.DataAs() : %w", err)
   		log.Printf("Error parsing proto payload: %s", err)
   		return err
   	}
   	payload := logentry.ProtoPayload
   	creator, ok := payload.AuthenticationInfo["principalEmail"]
   	if !ok {
   		err := fmt.Errorf("principalEmail not found in cloud event payload: %v", payload)
   		log.Printf("creator email not found: %s", err)
   		return err
   	}
   
   	// Get relevant VM instance details from the event's `subject` property
   	// Subject format:
   	// compute.googleapis.com/projects/<PROJECT>/zones/<ZONE>/instances/<INSTANCE>
   	paths := strings.Split(ev.Subject(), "/")
   	if len(paths) < 6 {
   		return fmt.Errorf("invalid event subject: %s", ev.Subject())
   	}
   	project := paths[2]
   	zone := paths[4]
   	instance := paths[6]
   
   	// Sanitize the `creator` label value to match GCE label requirements
   	// See https://cloud.google.com/compute/docs/labeling-resources#requirements
   	labelSanitizer := regexp.MustCompile("[^a-z0-9_-]+")
   	creatorstring := labelSanitizer.ReplaceAllString(strings.ToLower(creator.(string)), "_")
   
   	// Get the newly-created VM instance's label fingerprint
   	// This is a requirement of the Compute Engine API and avoids duplicate labels
   	inst, err := client.Get(ctx, &computepb.GetInstanceRequest{
   		Project:  project,
   		Zone:     zone,
   		Instance: instance,
   	})
   	if err != nil {
   		err = fmt.Errorf("could not retrieve GCE instance: %s", err)
   		log.Print(err)
   		return err
   	}
   	if v, ok := inst.Labels["creator"]; ok {
   		// Instance already has a creator label.
   		log.Printf("instance %s already labeled with creator: %s", instance, v)
   		return nil
   	}
   
   	// Add the creator label to the instance
   	op, err := client.SetLabels(ctx, &computepb.SetLabelsInstanceRequest{
   		Project:  project,
   		Zone:     zone,
   		Instance: instance,
   		InstancesSetLabelsRequestResource: &computepb.InstancesSetLabelsRequest{
   			LabelFingerprint: proto.String(inst.GetLabelFingerprint()),
   			Labels: map[string]string{
   				"creator": creatorstring,
   			},
   		},
   	})
   	if err != nil {
   		log.Fatalf("Could not label GCE instance: %s", err)
   	}
   	log.Printf("Creator label added to %s in operation %v", instance, op)
   	return nil
   }
   

   Java

   import com.google.cloud.compute.v1.GetInstanceRequest;
   import com.google.cloud.compute.v1.Instance;
   import com.google.cloud.compute.v1.InstancesClient;
   import com.google.cloud.compute.v1.InstancesSetLabelsRequest;
   import com.google.cloud.compute.v1.SetLabelsInstanceRequest;
   import com.google.cloud.functions.CloudEventsFunction;
   import com.google.gson.Gson;
   import com.google.gson.JsonObject;
   import com.google.gson.JsonSyntaxException;
   import io.cloudevents.CloudEvent;
   import java.nio.charset.StandardCharsets;
   import java.util.logging.Logger;
   
   public class AutoLabelInstance implements CloudEventsFunction {
     private static final Logger logger = Logger.getLogger(AutoLabelInstance.class.getName());
   
     @Override
     public void accept(CloudEvent event) throws Exception {
       // Extract CloudEvent data
       if (event.getData() != null) {
         String cloudEventData = new String(event.getData().toBytes(), StandardCharsets.UTF_8);
   
         // Convert data to JSON
         JsonObject eventData;
         try {
           Gson gson = new Gson();
           eventData = gson.fromJson(cloudEventData, JsonObject.class);
         } catch (JsonSyntaxException error) {
           throw new RuntimeException("CloudEvent data is not valid JSON: " + error.getMessage());
         }
   
         // Extract the Cloud Audit Logging entry from the data's protoPayload
         JsonObject payload = eventData.getAsJsonObject("protoPayload");
         JsonObject auth = payload.getAsJsonObject("authenticationInfo");
   
         // Extract the email address of the authenticated user
         // (or service account on behalf of third party principal) making the request
         String creator = auth.get("principalEmail").getAsString();
         if (creator == null) {
           throw new RuntimeException("`principalEmail` not found in protoPayload.");
         }
         // Format the 'creator' parameter to match GCE label validation requirements
         creator = creator.toLowerCase().replaceAll("\\W", "-");
   
         // Get relevant VM instance details from the CloudEvent `subject` property
         // Example: compute.googleapis.com/projects/<PROJECT>/zones/<ZONE>/instances/<INSTANCE>
         String subject = event.getSubject();
         if (subject == null || subject == "") {
           throw new RuntimeException("Missing CloudEvent `subject`.");
         }
         String[] params = subject.split("/");
   
         // Validate data
         if (params.length < 7) {
           throw new RuntimeException("Can not parse resource from CloudEvent `subject`: " + subject);
         }
         String project = params[2];
         String zone = params[4];
         String instanceName = params[6];
   
         // Instantiate the Compute Instances client
         try (InstancesClient instancesClient = InstancesClient.create()) {
           // Get the newly-created VM instance's label fingerprint
           // This is required by the Compute Engine API to prevent duplicate labels
           GetInstanceRequest getInstanceRequest =
               GetInstanceRequest.newBuilder()
                   .setInstance(instanceName)
                   .setProject(project)
                   .setZone(zone)
                   .build();
           Instance instance = instancesClient.get(getInstanceRequest);
           String fingerPrint = instance.getLabelFingerprint();
   
           // Label the instance with its creator
           SetLabelsInstanceRequest setLabelRequest =
               SetLabelsInstanceRequest.newBuilder()
                   .setInstance(instanceName)
                   .setProject(project)
                   .setZone(zone)
                   .setInstancesSetLabelsRequestResource(
                       InstancesSetLabelsRequest.newBuilder()
                           .putLabels("creator", creator)
                           .setLabelFingerprint(fingerPrint)
                           .build())
                   .build();
   
           instancesClient.setLabelsAsync(setLabelRequest);
           logger.info(
               String.format(
                   "Adding label, \"{'creator': '%s'}\", to instance, \"%s\".",
                   creator, instanceName));
         } catch (Exception error) {
           throw new RuntimeException(
               String.format(
                   "Error trying to label VM instance, %s: %s", instanceName, error.toString()));
         }
       }
     }
   }

Deploying the function

To deploy the function with a Cloud Audit Logs trigger, run the following command in the directory that contains the sample code (or in the case of Java, the pom.xml file):

Node.js

gcloud functions deploy nodejs-cal-function \
--gen2 \
--runtime=nodejs20 \
--region=REGION \
--source=. \
--entry-point=autoLabelInstance \
--trigger-location=REGION \
--trigger-event-filters="type=google.cloud.audit.log.v1.written" \
--trigger-event-filters="serviceName=compute.googleapis.com" \
--trigger-event-filters="methodName=v1.compute.instances.insert"

Use the --runtime flag to specify the runtime ID of a supported Node.js version to run your function.

Python

gcloud functions deploy python-cal-function \
--gen2 \
--runtime=python312 \
--region=REGION \
--source=. \
--entry-point=label_gce_instance \
--trigger-location=REGION \
--trigger-event-filters="type=google.cloud.audit.log.v1.written" \
--trigger-event-filters="serviceName=compute.googleapis.com" \
--trigger-event-filters="methodName=v1.compute.instances.insert"

Use the --runtime flag to specify the runtime ID of a supported Python version to run your function.

Go

gcloud functions deploy go-cal-function \
--gen2 \
--runtime=go121 \
--region=REGION \
--source=. \
--entry-point=label-gce-instance \
--trigger-location=REGION \
--trigger-event-filters="type=google.cloud.audit.log.v1.written" \
--trigger-event-filters="serviceName=compute.googleapis.com" \
--trigger-event-filters="methodName=v1.compute.instances.insert"

Use the --runtime flag to specify the runtime ID of a supported Go version to run your function.

Java

gcloud functions deploy java-cal-function \
--gen2 \
--runtime=java17 \
--region=REGION \
--source=. \
--entry-point=functions.AutoLabelInstance \
--memory=512MB \
--trigger-location=REGION \
--trigger-event-filters="type=google.cloud.audit.log.v1.written" \
--trigger-event-filters="serviceName=compute.googleapis.com" \
--trigger-event-filters="methodName=v1.compute.instances.insert"

Use the --runtime flag to specify the runtime ID of a supported Java version to run your function.

Note: You must include the --trigger-location flag if the resource you are monitoring for events is located in a region different from where you are deploying the function. Otherwise, the value you have set for region is used by default. See Eventarc triggers for details.

The deployment command above specifies the following event filter parameters that correspond to VM creation:

• type: The Cloud Audit Logs event type (google.cloud.audit.log.v1.written).
• serviceName: The name of the Google Cloud service that generated the log entry, in this case compute.googleapis.com.
• methodName: The name of the API method that generated the log entry, in this case v1.compute.instances.insert.

Triggering the function

Once the function is deployed, you can confirm that it works:

1. Create a Compute Engine VM instance:

   gcloud compute instances create YOUR_INSTANCE_NAME --zone YOUR_ZONE
   

   Alternatively, go to the Google Cloud console and click Create a VM.

2. Run the following command to verify that the instance has been labeled appropriately:

   gcloud compute instances describe YOUR_INSTANCE_NAME \
       --zone YOUR_ZONE \
       --format 'value(labels)'
   

   You should see a label with the format creator=YOURNAMEYOUR_DOMAIN.

Clean up

To avoid incurring charges to your Google Cloud account for the resources used in this tutorial, either delete the project that contains the resources, or keep the project and delete the individual resources.

Deleting the project

The easiest way to eliminate billing is to delete the project that you created for the tutorial.

To delete the project:

Caution: Deleting a project has the following effects:

• Everything in the project is deleted. If you used an existing project for the tasks in this document, when you delete it, you also delete any other work you've done in the project.
• Custom project IDs are lost. When you created this project, you might have created a custom project ID that you want to use in the future. To preserve the URLs that use the project ID, such as an appspot.com URL, delete selected resources inside the project instead of deleting the whole project.

If you plan to explore multiple architectures, tutorials, or quickstarts, reusing projects can help you avoid exceeding project quota limits.

1. In the Google Cloud console, go to the Manage resources page.

   Go to Manage resources

2. In the project list, select the project that you want to delete, and then click Delete.
3. In the dialog, type the project ID, and then click Shut down to delete the project.

Deleting the function

Deleting Cloud Run functions does not remove any resources stored in Cloud Storage.

To delete the Cloud Run function you created in this tutorial, run the following command:

Node.js

gcloud functions delete nodejs-cal-function --gen2 --region REGION 

Python

gcloud functions delete python-cal-function --gen2 --region REGION 

Go

gcloud functions delete go-cal-function --gen2 --region REGION 

Java

gcloud functions delete java-cal-function --gen2 --region REGION 

You can also delete Cloud Run functions from the Google Cloud console.

Deleting the Compute Engine VM instance

To delete the Compute Engine VM instance you created in this tutorial, run the following command:

gcloud compute instances delete YOUR_INSTANCE_NAME --zone YOUR_ZONE