Cloud Audit Logging 教程(第 2 代)


本教程演示了如何使用 Cloud Audit Logs 触发器编写、部署和触发事件驱动型 Cloud Functions 函数

Cloud Functions (第 2 代) 中的一项独特功能是,您的函数可以由 Cloud Audit Logs 条目触发。当发生重要的产品内操作时,许多 Google Cloud 产品都会向 Cloud Audit Logs 写入数据。这些日志条目可以实时触发 Cloud Functions 函数的执行,从而允许用户自动处理该函数和/或对其执行操作。

这些日志由 Google Cloud 中的许多不同事件生成,并且涵盖大多数 Google Cloud 产品。因此,Cloud Audit Logs 触发器使您能够创建函数来响应 Google Cloud 中的大多数状态更改。

本教程介绍了如何使用 Cloud Audit Logs 触发器为新创建的 Compute Engine 实例添加创建它们的实体(个人或服务账号)的名称标签。

如果您刚接触 Cloud Audit Logs,并且希望了解更多有关该产品的信息,请参阅 Cloud Audit Logs 文档

目标

  • 编写事件驱动型 Cloud Functions 函数,以便在创建 Compute Engine 虚拟机实例时接收 Cloud Audit Logs 事件。
  • 通过创建 Compute Engine 虚拟机实例来触发该函数,此时该实例标有创建它的实体(个人或服务账号)的名称。

费用

在本文档中,您将使用 Google Cloud 的以下收费组件:

  • Cloud Functions
  • Cloud Build
  • Pub/Sub
  • Artifact Registry
  • Eventarc
  • Cloud Logging
  • Compute Engine

For details, see Cloud Functions pricing.

您可使用价格计算器根据您的预计使用情况来估算费用。 Google Cloud 新用户可能有资格申请免费试用

准备工作

  1. 登录您的 Google Cloud 账号。如果您是 Google Cloud 新手,请创建一个账号来评估我们的产品在实际场景中的表现。新客户还可获享 $300 赠金,用于运行、测试和部署工作负载。
  2. 在 Google Cloud Console 中的项目选择器页面上,选择或创建一个 Google Cloud 项目

    转到“项目选择器”

  3. 确保您的 Google Cloud 项目已启用结算功能

  4. 启用 Cloud Functions, Cloud Run, Cloud Build, Artifact Registry, Eventarc, Logging, Compute Engine, and Pub/Sub API。

    启用 API

  5. 在 Google Cloud Console 中的项目选择器页面上,选择或创建一个 Google Cloud 项目

    转到“项目选择器”

  6. 确保您的 Google Cloud 项目已启用结算功能

  7. 启用 Cloud Functions, Cloud Run, Cloud Build, Artifact Registry, Eventarc, Logging, Compute Engine, and Pub/Sub API。

    启用 API

  8. 安装并初始化 Cloud SDK
  9. 更新 gcloud 组件:
  10. gcloud components update

    需要命令提示符吗?您可以使用 Google Cloud Shell。Google Cloud Shell 命令行环境已经包含 Google Cloud SDK,因此您无需再进行安装。Google Compute Engine 虚拟机也预装了 Google Cloud SDK。

  11. 准备开发环境。

前提条件

  1. 在 Google Cloud 控制台中打开 IAM 和管理 > 审核日志页面:

    打开“IAM 和管理”>“审核日志”页面

  2. 为 Compute Engine API 启用 Cloud Audit Logs 管理员读取数据读取数据写入日志类型:

    展示为 Compute Engine 启用审核日志的屏幕截图

  3. 检查 Compute Engine 服务账号是否具有 Editor 角色。此服务账号将用作 Cloud Functions 的服务身份。

    转到“IAM 和管理”>“IAM”页面

    找到表中的条目 PROJECT_NUMBER-compute@developer.gserviceaccount.com 并查看 Roles 列。如果列包含 Editor,则可以跳过以下步骤。否则,请转到下一步,并将必要的角色分配给服务账号。

  4. 向项目的 Compute Engine 服务账号授予 eventarc.eventReceiver 角色:

    PROJECT_ID=$(gcloud config get-value project)
    PROJECT_NUMBER=$(gcloud projects list --filter="project_id:$PROJECT_ID" --format='value(project_number)')
    
    # Allow service account token creation
    gcloud projects add-iam-policy-binding $PROJECT_ID \
     --member serviceAccount:$PROJECT_NUMBER-compute@developer.gserviceaccount.com \
     --role roles/eventarc.eventReceiver
    
  5. run.invoker 角色授予项目的 Compute Engine 服务账号,以便 Pub/Sub 触发器可以执行该函数:

    PROJECT_ID=$(gcloud config get-value project)
    PROJECT_NUMBER=$(gcloud projects list --filter="project_id:$PROJECT_ID" --format='value(project_number)')
    
    # Allow service account token creation
    gcloud projects add-iam-policy-binding $PROJECT_ID \
     --member serviceAccount:$PROJECT_NUMBER-compute@developer.gserviceaccount.com \
     --role roles/run.invoker
    
  6. compute.instanceAdmin 角色授予项目的 Compute Engine 服务账号,以便函数代码具有获取虚拟机实例并为其设置标签所需的权限:

    PROJECT_ID=$(gcloud config get-value project)
    PROJECT_NUMBER=$(gcloud projects list --filter="project_id:$PROJECT_ID" --format='value(project_number)')
    
    # Allow service account token creation
    gcloud projects add-iam-policy-binding $PROJECT_ID \
     --member serviceAccount:$PROJECT_NUMBER-compute@developer.gserviceaccount.com \
     --role roles/compute.instanceAdmin
    

准备应用

  1. 将示例应用代码库克隆到本地机器:

    Node.js

    git clone https://github.com/GoogleCloudPlatform/nodejs-docs-samples.git

    或者,您也可以下载该示例的 zip 文件并将其解压缩。

    Python

    git clone https://github.com/GoogleCloudPlatform/python-docs-samples.git

    或者,您也可以下载该示例的 zip 文件并将其解压缩。

    Go

    git clone https://github.com/GoogleCloudPlatform/golang-samples.git

    或者,您也可以下载该示例的 zip 文件并将其解压缩。

    Java

    git clone https://github.com/GoogleCloudPlatform/java-docs-samples.git

    或者,您也可以下载该示例的 zip 文件并将其解压缩。

  2. 切换到 Cloud Functions 示例代码所在的目录,以便访问 Cloud Audit Logs:

    Node.js

    cd nodejs-docs-samples/functions/v2/autoLabelInstance/

    Python

    cd python-docs-samples/functions/v2/label_gce_instance/

    Go

    cd golang-samples/functions/functionsv2/label_gce_instance/

    Java

    cd java-docs-samples/functions/v2/label-compute-instance/

  3. 查看示例代码:

    Node.js

    const functions = require('@google-cloud/functions-framework');
    
    const compute = require('@google-cloud/compute');
    const instancesClient = new compute.InstancesClient();
    
    // Register a CloudEvent callback with the Functions Framework that labels
    // newly-created GCE instances with the entity (person or service account)
    // that created them.
    functions.cloudEvent('autoLabelInstance', async cloudEvent => {
      // Extract parameters from the CloudEvent + Cloud Audit Log data
      const payload = cloudEvent.data && cloudEvent.data.protoPayload;
      const authInfo = payload && payload.authenticationInfo;
      let creator = authInfo && authInfo.principalEmail;
    
      // Get relevant VM instance details from the CloudEvent's `subject` property
      // Example value:
      //   compute.googleapis.com/projects/<PROJECT>/zones/<ZONE>/instances/<INSTANCE>
      const params = cloudEvent.subject && cloudEvent.subject.split('/');
    
      // Validate data
      if (!creator || !params || params.length !== 7) {
        throw new Error('Invalid event structure');
      }
    
      // Format the 'creator' parameter to match GCE label validation requirements
      creator = creator.toLowerCase().replace(/\W/g, '_');
    
      // Get the newly-created VM instance's label fingerprint
      // This is required by the Compute Engine API to prevent duplicate labels
      const getInstanceRequest = {
        project: params[2],
        zone: params[4],
        instance: params[6],
      };
      const [instance] = await instancesClient.get(getInstanceRequest);
    
      // Label the instance with its creator
      const setLabelsRequest = Object.assign(
        {
          instancesSetLabelsRequestResource: {
            labels: {creator},
            labelFingerprint: instance.labelFingerprint,
          },
        },
        getInstanceRequest
      );
    
      return instancesClient.setLabels(setLabelsRequest);
    });

    Python

    import re
    
    from google.api_core.exceptions import GoogleAPIError
    from google.cloud import compute_v1
    from google.cloud.compute_v1.types import compute
    
    instances_client = compute_v1.InstancesClient()
    
    
    # CloudEvent function that labels newly-created GCE instances
    # with the entity (user or service account) that created them.
    #
    # @param {object} cloudevent A CloudEvent containing the Cloud Audit Log entry.
    # @param {object} cloudevent.data.protoPayload The Cloud Audit Log entry.
    def label_gce_instance(cloudevent):
        # Extract parameters from the CloudEvent + Cloud Audit Log data
        payload = cloudevent.data.get("protoPayload", dict())
        auth_info = payload.get("authenticationInfo", dict())
        creator = auth_info.get("principalEmail")
    
        # Get relevant VM instance details from the cloudevent's `subject` property
        # Example value:
        #   compute.googleapis.com/projects/<PROJECT_ID>/zones/<ZONE_ID>/instances/<INSTANCE_NAME>
        instance_params = cloudevent["subject"].split("/")
    
        # Validate data
        if not creator or not instance_params or len(instance_params) != 7:
            # This is not something retries will fix, so don't throw an Exception
            # (Thrown exceptions trigger retries *if* you enable retries in GCF.)
            print("ERROR: Invalid `principalEmail` and/or CloudEvent `subject`.")
            return
    
        instance_project = instance_params[2]
        instance_zone = instance_params[4]
        instance_name = instance_params[6]
    
        # Format the 'creator' parameter to match GCE label validation requirements
        creator = re.sub("\\W", "_", creator.lower())
    
        # Get the newly-created VM instance's label fingerprint
        # This is required by the Compute Engine API to prevent duplicate labels
        instance = instances_client.get(
            project=instance_project, zone=instance_zone, instance=instance_name
        )
    
        # Construct API call to label the VM instance with its creator
        request_init = {
            "project": instance_project,
            "zone": instance_zone,
            "instance": instance_name,
        }
        request_init[
            "instances_set_labels_request_resource"
        ] = compute.InstancesSetLabelsRequest(
            label_fingerprint=instance.label_fingerprint, labels={"creator": creator}
        )
        request = compute.SetLabelsInstanceRequest(request_init)
    
        # Perform instance-labeling API call
        try:
            instances_client.set_labels_unary(request)
            print(f"Labelled VM instance {instance_name} with creator: {creator}")
        except GoogleAPIError as e:
            # Swallowing the exception means failed invocations WON'T be retried
            print("Label operation failed", e)
    
            # Uncomment the line below to retry failed invocations.
            # (You'll also have to enable retries in Cloud Functions itself.)
            # raise e
    
        return
    
    

    Go

    
    // Package helloworld provides a set of Cloud Functions samples.
    package helloworld
    
    import (
    	"context"
    	"fmt"
    	"log"
    	"regexp"
    	"strings"
    
    	compute "cloud.google.com/go/compute/apiv1"
    	"github.com/GoogleCloudPlatform/functions-framework-go/functions"
    	"github.com/cloudevents/sdk-go/v2/event"
    	computepb "google.golang.org/genproto/googleapis/cloud/compute/v1"
    	"google.golang.org/protobuf/proto"
    )
    
    // AuditLogEntry represents a LogEntry as described at
    // https://cloud.google.com/logging/docs/reference/v2/rest/v2/LogEntry
    type AuditLogEntry struct {
    	ProtoPayload *AuditLogProtoPayload `json:"protoPayload"`
    }
    
    // AuditLogProtoPayload represents AuditLog within the LogEntry.protoPayload
    // See https://cloud.google.com/logging/docs/reference/audit/auditlog/rest/Shared.Types/AuditLog
    type AuditLogProtoPayload struct {
    	MethodName         string                 `json:"methodName"`
    	ResourceName       string                 `json:"resourceName"`
    	AuthenticationInfo map[string]interface{} `json:"authenticationInfo"`
    }
    
    var client *compute.InstancesClient
    
    func init() {
    	// Create an Instances Client
    	var err error
    	client, err = compute.NewInstancesRESTClient(context.Background())
    	if err != nil {
    		log.Fatalf("Failed to create instances client: %s", err)
    	}
    
    	functions.CloudEvent("label-gce-instance", labelGceInstance)
    }
    
    // Cloud Function that receives GCE instance creation Audit Logs, and adds a
    // `creator` label to the instance.
    func labelGceInstance(ctx context.Context, ev event.Event) error {
    	// Extract parameters from the Cloud Event and Cloud Audit Log data
    	logentry := &AuditLogEntry{}
    	if err := ev.DataAs(logentry); err != nil {
    		err = fmt.Errorf("event.DataAs() : %w", err)
    		log.Printf("Error parsing proto payload: %s", err)
    		return err
    	}
    	payload := logentry.ProtoPayload
    	creator, ok := payload.AuthenticationInfo["principalEmail"]
    	if !ok {
    		err := fmt.Errorf("principalEmail not found in cloud event payload: %v", payload)
    		log.Printf("creator email not found: %s", err)
    		return err
    	}
    
    	// Get relevant VM instance details from the event's `subject` property
    	// Subject format:
    	// compute.googleapis.com/projects/<PROJECT>/zones/<ZONE>/instances/<INSTANCE>
    	paths := strings.Split(ev.Subject(), "/")
    	if len(paths) < 6 {
    		return fmt.Errorf("invalid event subject: %s", ev.Subject())
    	}
    	project := paths[2]
    	zone := paths[4]
    	instance := paths[6]
    
    	// Sanitize the `creator` label value to match GCE label requirements
    	// See https://cloud.google.com/compute/docs/labeling-resources#requirements
    	labelSanitizer := regexp.MustCompile("[^a-z0-9_-]+")
    	creatorstring := labelSanitizer.ReplaceAllString(strings.ToLower(creator.(string)), "_")
    
    	// Get the newly-created VM instance's label fingerprint
    	// This is a requirement of the Compute Engine API and avoids duplicate labels
    	inst, err := client.Get(ctx, &computepb.GetInstanceRequest{
    		Project:  project,
    		Zone:     zone,
    		Instance: instance,
    	})
    	if err != nil {
    		err = fmt.Errorf("could not retrieve GCE instance: %s", err)
    		log.Print(err)
    		return err
    	}
    	if v, ok := inst.Labels["creator"]; ok {
    		// Instance already has a creator label.
    		log.Printf("instance %s already labeled with creator: %s", instance, v)
    		return nil
    	}
    
    	// Add the creator label to the instance
    	op, err := client.SetLabels(ctx, &computepb.SetLabelsInstanceRequest{
    		Project:  project,
    		Zone:     zone,
    		Instance: instance,
    		InstancesSetLabelsRequestResource: &computepb.InstancesSetLabelsRequest{
    			LabelFingerprint: proto.String(inst.GetLabelFingerprint()),
    			Labels: map[string]string{
    				"creator": creatorstring,
    			},
    		},
    	})
    	if err != nil {
    		log.Fatalf("Could not label GCE instance: %s", err)
    	}
    	log.Printf("Creator label added to %s in operation %v", instance, op)
    	return nil
    }
    

    Java

    import com.google.cloud.compute.v1.GetInstanceRequest;
    import com.google.cloud.compute.v1.Instance;
    import com.google.cloud.compute.v1.InstancesClient;
    import com.google.cloud.compute.v1.InstancesSetLabelsRequest;
    import com.google.cloud.compute.v1.SetLabelsInstanceRequest;
    import com.google.cloud.functions.CloudEventsFunction;
    import com.google.gson.Gson;
    import com.google.gson.JsonObject;
    import com.google.gson.JsonSyntaxException;
    import io.cloudevents.CloudEvent;
    import java.nio.charset.StandardCharsets;
    import java.util.logging.Logger;
    
    public class AutoLabelInstance implements CloudEventsFunction {
      private static final Logger logger = Logger.getLogger(AutoLabelInstance.class.getName());
    
      @Override
      public void accept(CloudEvent event) throws Exception {
        // Extract CloudEvent data
        if (event.getData() != null) {
          String cloudEventData = new String(event.getData().toBytes(), StandardCharsets.UTF_8);
    
          // Convert data to JSON
          JsonObject eventData;
          try {
            Gson gson = new Gson();
            eventData = gson.fromJson(cloudEventData, JsonObject.class);
          } catch (JsonSyntaxException error) {
            throw new RuntimeException("CloudEvent data is not valid JSON: " + error.getMessage());
          }
    
          // Extract the Cloud Audit Logging entry from the data's protoPayload
          JsonObject payload = eventData.getAsJsonObject("protoPayload");
          JsonObject auth = payload.getAsJsonObject("authenticationInfo");
    
          // Extract the email address of the authenticated user
          // (or service account on behalf of third party principal) making the request
          String creator = auth.get("principalEmail").getAsString();
          if (creator == null) {
            throw new RuntimeException("`principalEmail` not found in protoPayload.");
          }
          // Format the 'creator' parameter to match GCE label validation requirements
          creator = creator.toLowerCase().replaceAll("\\W", "-");
    
          // Get relevant VM instance details from the CloudEvent `subject` property
          // Example: compute.googleapis.com/projects/<PROJECT>/zones/<ZONE>/instances/<INSTANCE>
          String subject = event.getSubject();
          if (subject == null || subject == "") {
            throw new RuntimeException("Missing CloudEvent `subject`.");
          }
          String[] params = subject.split("/");
    
          // Validate data
          if (params.length < 7) {
            throw new RuntimeException("Can not parse resource from CloudEvent `subject`: " + subject);
          }
          String project = params[2];
          String zone = params[4];
          String instanceName = params[6];
    
          // Instantiate the Compute Instances client
          try (InstancesClient instancesClient = InstancesClient.create()) {
            // Get the newly-created VM instance's label fingerprint
            // This is required by the Compute Engine API to prevent duplicate labels
            GetInstanceRequest getInstanceRequest =
                GetInstanceRequest.newBuilder()
                    .setInstance(instanceName)
                    .setProject(project)
                    .setZone(zone)
                    .build();
            Instance instance = instancesClient.get(getInstanceRequest);
            String fingerPrint = instance.getLabelFingerprint();
    
            // Label the instance with its creator
            SetLabelsInstanceRequest setLabelRequest =
                SetLabelsInstanceRequest.newBuilder()
                    .setInstance(instanceName)
                    .setProject(project)
                    .setZone(zone)
                    .setInstancesSetLabelsRequestResource(
                        InstancesSetLabelsRequest.newBuilder()
                            .putLabels("creator", creator)
                            .setLabelFingerprint(fingerPrint)
                            .build())
                    .build();
    
            instancesClient.setLabelsAsync(setLabelRequest);
            logger.info(
                String.format(
                    "Adding label, \"{'creator': '%s'}\", to instance, \"%s\".",
                    creator, instanceName));
          } catch (Exception error) {
            throw new RuntimeException(
                String.format(
                    "Error trying to label VM instance, %s: %s", instanceName, error.toString()));
          }
        }
      }
    }

部署函数

如需使用 Cloud Audit Logs 触发器部署函数,请在包含示例代码(如果是 Java 则为 pom.xml 文件)的目录中运行以下命令:

Node.js

gcloud functions deploy nodejs-cal-function \
--gen2 \
--runtime=nodejs20 \
--region=REGION \
--source=. \
--entry-point=autoLabelInstance \
--trigger-location=REGION \
--trigger-event-filters="type=google.cloud.audit.log.v1.written" \
--trigger-event-filters="serviceName=compute.googleapis.com" \
--trigger-event-filters="methodName=v1.compute.instances.insert"

使用 --runtime 标志可以指定支持的 Node.js 版本的运行时 ID 来运行您的函数。

Python

gcloud functions deploy python-cal-function \
--gen2 \
--runtime=python312 \
--region=REGION \
--source=. \
--entry-point=label_gce_instance \
--trigger-location=REGION \
--trigger-event-filters="type=google.cloud.audit.log.v1.written" \
--trigger-event-filters="serviceName=compute.googleapis.com" \
--trigger-event-filters="methodName=v1.compute.instances.insert"

使用 --runtime 标志可以指定支持的 Python 版本的运行时 ID 来运行您的函数。

Go

gcloud functions deploy go-cal-function \
--gen2 \
--runtime=go121 \
--region=REGION \
--source=. \
--entry-point=label-gce-instance \
--trigger-location=REGION \
--trigger-event-filters="type=google.cloud.audit.log.v1.written" \
--trigger-event-filters="serviceName=compute.googleapis.com" \
--trigger-event-filters="methodName=v1.compute.instances.insert"

使用 --runtime 标志可以指定支持的 Go 版本的运行时 ID 来运行您的函数。

Java

gcloud functions deploy java-cal-function \
--gen2 \
--runtime=java17 \
--region=REGION \
--source=. \
--entry-point=functions.AutoLabelInstance \
--memory=512MB \
--trigger-location=REGION \
--trigger-event-filters="type=google.cloud.audit.log.v1.written" \
--trigger-event-filters="serviceName=compute.googleapis.com" \
--trigger-event-filters="methodName=v1.compute.instances.insert"

使用 --runtime 标志可以指定支持的 Java 版本的运行时 ID 来运行您的函数。

上面的部署命令指定了与所创建的虚拟机对应的以下事件过滤条件参数:

  • type:Cloud Audit Logs 事件类型 (google.cloud.audit.log.v1.written)。
  • serviceName:生成日志条目的 Google Cloud 服务的名称,在本示例中为 compute.googleapis.com
  • methodName:生成日志条目的 API 方法的名称,在本示例中为 v1.compute.instances.insert

触发函数

函数部署后,您可以确认其是否正常运行:

  1. 创建 Compute Engine 虚拟机实例:

    gcloud compute instances create YOUR_INSTANCE_NAME --zone YOUR_ZONE
    

    或者,转到 Google Cloud 控制台,然后点击创建虚拟机

  2. 运行以下命令来验证实例是否已正确添加标签:

    gcloud compute instances describe YOUR_INSTANCE_NAME \
        --zone YOUR_ZONE \
        --format 'value(labels)'
    

    您应该会看到格式为 creator=YOURNAMEYOUR_DOMAIN 的标签。

清理

为避免因本教程中使用的资源导致您的 Google Cloud 账号产生费用,请删除包含这些资源的项目,或者保留项目但删除各个资源。

删除项目

为了避免产生费用,最简单的方法是删除您为本教程创建的项目。

要删除项目,请执行以下操作:

  1. 在 Google Cloud 控制台中,进入管理资源页面。

    转到“管理资源”

  2. 在项目列表中,选择要删除的项目,然后点击删除
  3. 在对话框中输入项目 ID,然后点击关闭以删除项目。

删除 Cloud Functions 函数

删除 Cloud Functions 函数不会移除存储在 Cloud Storage 中的任何资源。

如需删除您在本教程中创建的 Cloud Functions 函数,请运行以下命令:

Node.js

gcloud functions delete nodejs-cal-function --gen2 --region REGION 

Python

gcloud functions delete python-cal-function --gen2 --region REGION 

Go

gcloud functions delete go-cal-function --gen2 --region REGION 

Java

gcloud functions delete java-cal-function --gen2 --region REGION 

您也可以通过 Google Cloud 控制台删除 Cloud Functions 函数。

删除 Compute Engine 虚拟机实例

如需删除您在本教程中创建的 Compute Engine 虚拟机实例,请运行以下命令:

gcloud compute instances delete YOUR_INSTANCE_NAME --zone YOUR_ZONE