Second-Party Triggers with Cloud Logging

Many Google Cloud events are logged in Cloud Audit Logs. You can filter these logs and forward them to Pub/Sub topics using sinks. These Pub/Sub topics can then send notifications that trigger Cloud Run functions. This lets you create custom events from any Google Cloud service that produces audit logs.

Configuration

To run the sample in this document, you'll need a Pub/Sub topic and a Cloud Logging sink. The sample uses them to forward Cloud Audit Logs to a Cloud Run function.

Event structure

Like all Pub/Sub-triggered functions, functions triggered by Cloud log entries receive a PubsubMessage object whose data parameter is a base64-encoded string. For Cloud log events, decoding this value returns the relevant log entry as a JSON string.

Sample code

You can use a Pub/Sub-triggered function to detect and respond to exported Cloud logs:

Node.js

exports.processLogEntry = data => {
  const dataBuffer = Buffer.from(data.data, 'base64');

  const logEntry = JSON.parse(dataBuffer.toString('ascii')).protoPayload;
  console.log(`Method: ${logEntry.methodName}`);
  console.log(`Resource: ${logEntry.resourceName}`);
  console.log(`Initiator: ${logEntry.authenticationInfo.principalEmail}`);
};

Python

import base64
import json

def process_log_entry(data, context):
    data_buffer = base64.b64decode(data["data"])
    log_entry = json.loads(data_buffer)["protoPayload"]

    print(f"Method: {log_entry['methodName']}")
    print(f"Resource: {log_entry['resourceName']}")
    print(f"Initiator: {log_entry['authenticationInfo']['principalEmail']}")

Go

// Package log contains examples for handling Cloud Functions logs.
package log

import (
	"context"
	"log"
)

// PubSubMessage is the payload of a Pub/Sub event.
// See the documentation for more details:
// https://cloud.google.com/pubsub/docs/reference/rest/v1/PubsubMessage
type PubSubMessage struct {
	Data []byte `json:"data"`
}

// ProcessLogEntry processes a Pub/Sub message from Cloud Logging.
func ProcessLogEntry(ctx context.Context, m PubSubMessage) error {
	log.Printf("Log entry data: %s", string(m.Data))
	return nil
}

Java

import com.google.cloud.functions.BackgroundFunction;
import com.google.cloud.functions.Context;
import functions.eventpojos.PubsubMessage;
import java.nio.charset.StandardCharsets;
import java.util.Base64;
import java.util.logging.Logger;

public class StackdriverLogging implements BackgroundFunction<PubsubMessage> {
  private static final Logger logger = Logger.getLogger(StackdriverLogging.class.getName());

  @Override
  public void accept(PubsubMessage message, Context context) {
    String name = "World";

    if (!message.getData().isEmpty()) {
      name = new String(Base64.getDecoder().decode(
          message.getData().getBytes(StandardCharsets.UTF_8)), StandardCharsets.UTF_8);
    }
    String res = String.format("Hello, %s", name);
    logger.info(res);
  }
}

Deploying a function

Use the following command to deploy the function:

Node.js

gcloud functions deploy processLogEntry \
--runtime nodejs20 \
--trigger-topic YOUR_PUBSUB_TOPIC/
FLAGS...

Use the --runtime flag to specify the runtime ID of a supported Node.js version to run your function.

Python

gcloud functions deploy process_log_entry \
--runtime python312 \
--trigger-topic YOUR_PUBSUB_TOPIC/
FLAGS...

Use the --runtime flag to specify the runtime ID of a supported Python version to run your function.

Go

gcloud functions deploy ProcessLogEntry \
--runtime go121 \
--trigger-topic YOUR_PUBSUB_TOPIC/
FLAGS...

Use the --runtime flag to specify the runtime ID of a supported Go version to run your function.

Java

gcloud functions deploy java-log-function \
--entry-point StackdriverLogging \
--runtime java17 \
--memory 512MB \
--trigger-topic YOUR_PUBSUB_TOPIC/
FLAGS...

Use the --runtime flag to specify the runtime ID of a supported Java version to run your function.

Triggering a function

When a Cloud log entry that matches one of your filters is created, you should see corresponding log entries for your function:

Method: METHOD
Resource: projects/YOUR_GCLOUD_PROJECT/...
Initiator: YOUR_EMAIL_ADDRESS