View logs routed to Pub/Sub

This document explains how you can find log entries that you routed from Cloud Logging to Pub/Sub topics, which occurs in near real-time. We recommend using Pub/Sub for integrating Cloud Logging logs with third-party software.

When you route logs to a Pub/Sub topic, Logging publishes each log entry as a Pub/Sub message as soon as Logging receives that log entry. Routed logs are generally available within seconds of their arrival to Logging, with 99% of logs available in less than 60 seconds.

Before you begin

For a conceptual discussion of sinks, see Overview of routing and storage models: Sinks.

For instructions on how to route your logs, see Route logs to supported destinations.

View logs

To view your logs as they are streamed through Pub/Sub, do the following:

  1. In the Google Cloud console, select Pub/Sub, and then select Topics, or click the following button:

    Go to Pub/Sub Topics

  2. Find or create a subscription to the topic used in the log sink, and pull a log entry from it. You might have to wait for a new log entry to be published.

Logs organization

The data field of each message is a base64-encoded LogEntry object. As an example, a Pub/Sub subscriber might pull the following object from a topic that is receiving log entries. The object shown contains a list with a single message, although Pub/Sub might return several messages if several log entries are available. The data value (about 600 characters) and the ackId value (about 200 characters) have been shortened to make the example easier to read:

 "receivedMessages": [
   "message": {
    "data": "eyJtZXRhZGF0YSI6eyJzZXZ0eSI6Il...Dk0OTU2G9nIjoiaGVsbG93b3JsZC5sb2cifQ==",
    "attributes": {
     "": "instance",
     "": "123456"
    "messageId": "43913662360"

If you decode the data field and format it, you get the following LogEntry object:

  "log": "helloworld.log",
  "insertId": "2015-04-15|11:41:00.577447-07||-1694494956",
  "textPayload": "Wed Apr 15 20:40:51 CEST 2015 Hello, world!",
  "timestamp": "2015-04-15T18:40:56Z",
  "labels": {
    "\/resource_type": "instance",
    "\/resource_id": "123456"
  "severity": "WARNING"

Third-party integration with Pub/Sub

Logging supports logging integration with third parties, such as Splunk. For a current list of integrations, see Partners for Google Cloud's operations suite integrations.

You route your logs through a Pub/Sub topic and the third party receives your logs by subscribing to the same topic.

To perform the integration, expect to do something like the following:

  1. Obtain from the third party a Google Cloud service account name created from their Google Cloud project. For example, You use this name to give the third party permission to receive your logs.

  2. In your project containing the logs, enable the Pub/Sub API.

  3. Enable the Pub/Sub API.

    Enable the API

  4. Create a Pub/Sub topic. You can create a topic when you configure a log sink, or by following these steps:

    1. In the Google Cloud console, select Pub/Sub, and then select Topics, or click the following button:

      Go to Pub/Sub Topics

    2. Select Create topic and enter a topic name. For example, projects/my-project-id/topics/my-pubsub-topic. You route your logs to this topic.

      Each message sent to the topic include the timestamp of the routed log entry in the Pub/Sub message attributes; for example:

      "attributes": {
        "": "2018-10-01T00:00:00Z"
    3. Click Create topic.

    4. Authorize Logging to route logs to the topic. For instructions, see Set destination permissions.

  5. Authorize the third party to subscribe to your topic:

    1. In the Google Cloud console, select Pub/Sub, and then select Topics, or click the following button:

      Go to Pub/Sub Topics

    2. Select your topic.
    3. Select Permissions.
    4. Select Add permission and enter the third party's service account name.
    5. In the Select a role menu, select Pub/Sub Subscriber.
    6. Click Save.
  6. Provide the third party with the name of your Pub/Sub topic; for example, projects/my-project-number/topics/my-pubsub-topic. They should subscribe to the topic before you start routing.

  7. Start routing the logs after your third party has subscribed to the topic:

    1. In the Google Cloud console, select Logging, and then select Log Router, or click the following button:

      Go to the Log Router

    2. In your project containing the logs you want to route, click Create Sink.
    3. Enter a name and description for the sink, and then click Next.
    4. In the Sink Service menu, select Cloud Pub/Sub topic.
    5. In the Cloud Pub/Sub topic menu, select the Pub/Sub topic to which the third party is subscribed, and then click Next.
    6. Select the logs to include in the sink and then click Next.
    7. Optional: Select the logs to exclude.
    8. Click Create Sink.

      A dialog with the message Sink created appears. This message indicates that your sink was successfully created with permissions to write future matching logs to the destination you selected.

Your third party should begin receiving the log entries right away.

For an exploration of common logs routing scenarios using Pub/Sub, see Scenarios for exporting Cloud Logging data: Splunk.


If logs seem to be missing from your sink's destination or you otherwise suspect that your sink isn't properly routing logs, then see Troubleshoot routing logs.


Cloud Logging doesn't charge to route logs to a supported destination; however, the destination might apply charges. For information about destination costs, see Pub/Sub pricing.

If you send and then exclude your Virtual Private Cloud flow logs from Cloud Logging, then VPC flow log generation charges apply in addition to the destination charges.