Second-Party Triggers with Cloud Logging

Many Google Cloud Platform events are logged in Cloud Audit Logs. You can filter these logs and forward them to Pub/Sub topics using sinks. These Pub/Sub topics can then send notifications that trigger Cloud Functions. This allows you to create custom events from any Google Cloud Platform service that produces audit logs.

Configuration

To run the sample below, you'll need a Pub/Sub topic and a Cloud Logging sink. The sample uses them to forward Cloud Audit Logs to a Cloud Function.

Event structure

Like all Pub/Sub-triggered functions, functions triggered by Cloud log entries receive a PubsubMessage object whose data parameter is a base64-encoded string. For Cloud log events, decoding this value returns the relevant log entry as a JSON string.

Sample code

You can use a Pub/Sub-triggered function like the one below to detect and respond to exported Cloud logs:

Node.js

functions/log/index.js 
exports.processLogEntry = data => {
  const dataBuffer = Buffer.from(data.data, 'base64');

  const logEntry = JSON.parse(dataBuffer.toString('ascii')).protoPayload;
  console.log(`Method: ${logEntry.methodName}`);
  console.log(`Resource: ${logEntry.resourceName}`);
  console.log(`Initiator: ${logEntry.authenticationInfo.principalEmail}`);
};

Python

functions/log/main.py 
import base64
import json

def process_log_entry(data, context):
    data_buffer = base64.b64decode(data['data'])
    log_entry = json.loads(data_buffer)['protoPayload']

    print(f"Method: {log_entry['methodName']}")
    print(f"Resource: {log_entry['resourceName']}")
    print(f"Initiator: {log_entry['authenticationInfo']['principalEmail']}")

Go

functions/log/process_log_entry.go 

// Package log contains examples for handling Cloud Functions logs.
package log

import (
	"context"
	"log"
)

// PubSubMessage is the payload of a Pub/Sub event.
// See the documentation for more details:
// https://cloud.google.com/pubsub/docs/reference/rest/v1/PubsubMessage
type PubSubMessage struct {
	Data []byte `json:"data"`
}

// ProcessLogEntry processes a Pub/Sub message from Cloud Logging.
func ProcessLogEntry(ctx context.Context, m PubSubMessage) error {
	log.Printf("Log entry data: %s", string(m.Data))
	return nil
}

Java

functions/logging/stackdriver-logging/src/main/java/functions/StackdriverLogging.java 

import com.google.cloud.functions.BackgroundFunction;
import com.google.cloud.functions.Context;
import com.google.events.cloud.pubsub.v1.Message;
import java.nio.charset.StandardCharsets;
import java.util.Base64;
import java.util.logging.Logger;

public class StackdriverLogging implements BackgroundFunction<Message> {
  private static final Logger logger = Logger.getLogger(StackdriverLogging.class.getName());

  @Override
  public void accept(Message message, Context context) {
    String name = "World";

    if (!message.getData().isEmpty()) {
      name = new String(Base64.getDecoder().decode(
          message.getData().getBytes(StandardCharsets.UTF_8)), StandardCharsets.UTF_8);
    }
    String res = String.format("Hello, %s", name);
    logger.info(res);
  }
}

Deploying a function

Use the command below to deploy the function:

Node.js 

gcloud functions deploy processLogEntry \
--runtime nodejs14 \
--trigger-topic YOUR_PUBSUB_TOPIC
You can use the following values for the --runtime flag to specify your preferred Node.js version:
  • nodejs16 (preview)
  • nodejs14 (recommended)
  • nodejs12
  • nodejs10

Python 

gcloud functions deploy process_log_entry \
--runtime python39 \
--trigger-topic YOUR_PUBSUB_TOPIC
You can use the following values for the --runtime flag to specify your preferred Python version:
  • python39 (recommended)
  • python38
  • python37

Go 

gcloud functions deploy ProcessLogEntry \
--runtime go113 \
--trigger-topic YOUR_PUBSUB_TOPIC
You can use the following values for the --runtime flag to specify your preferred Go version:
  • go116 (preview)
  • go113 (recommended)
  • go111

Java 

gcloud functions deploy java-log-function \
--entry-point StackdriverLogging \
--runtime java11 \
--memory 512MB \
--trigger-topic YOUR_PUBSUB_TOPIC

Triggering a function

When a Cloud log entry that matches one of your filters is created, you should see corresponding log entries for your function:

Method: METHOD
Resource: projects/YOUR_GCLOUD_PROJECT/...
Initiator: YOUR_EMAIL_ADDRESS