Write and query log entries with the gcloud CLI

This document introduces you to some of the capabilities of Cloud Logging and shows you how to do the following:

  • Write log entries by using the Google Cloud CLI.
  • List log entries by using the gcloud CLI.
  • List log entries by using the Logging API.
  • View and query log entries by using the Logs Explorer.

Before you begin

You must have a Google Cloud project with billing enabled to complete this quickstart. If you don't have a Google Cloud project, or if you don't have billing enabled for your Google Cloud project, then do the following:
  1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.
  2. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

    Go to project selector

  3. Make sure that billing is enabled for your Google Cloud project.

  4. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

    Go to project selector

  5. Make sure that billing is enabled for your Google Cloud project.

Getting started

The gcloud CLI has a group of commands, gcloud logging, that provide a command-line interface to the Cloud Logging API.

You can use the Cloud Shell environment or a Compute Engine virtual machine (VM) instance for the gcloud CLI commands in this quickstart. The gcloud CLI is pre-installed in the Cloud Shell environment.

Cloud Shell

Verify the gcloud CLI is configured to use the correct Google Cloud project:

  1. From the Google Cloud console, click Activate Cloud Shell:

    Screenshot of Cloud Shell button in Google Cloud console.

    The Cloud Shell opens in a window and displays a welcome message. The welcome message echoes the configured project ID:

    Screenshot of Cloud Shell displaying a welcome message.

  2. If you want to use a different Google Cloud project than the one listed in the welcome message, then run the following command after replacing PROJECT_ID with your project ID:

       gcloud config set project PROJECT_ID
       

    To get the project ID, see Identifying projects.

VM instance

To create a Compute Engine VM instance in the Google Cloud console, follow these steps:

  1. From the Google Cloud console, select Compute Engine and then select VM instances.

  2. Select Create instance.

  3. In Identity and API access under Access scopes, select Set access for each API.

  4. Scroll through the APIs until you find the Stackdriver Logging API. Toggle the access to Full.

  5. Leave all other settings at their default values and click Create. Your VM instance is ready to use.

  6. Click SSH to connect to your VM instance shell. After a moment, a Debian GNU/Linux shell opens in a window and displays a welcome message.

  7. Run the following command to verify that gcloud CLI is configured for your Compute Engine project:

    gcloud config list
    
  8. If you want to use a different Google Cloud project, then run the following command after replacing PROJECT_ID with your project ID:

      gcloud config set project PROJECT_ID
      

    To get the project ID, see Identifying projects.

Write log entries by using the gcloud CLI

Logging supports log entries with structured and unstructured data. Structured data consists of a JSON data structure; for example, {"weather": "partly cloudy"}. Unstructured data is a string of characters; for example, "A simple entry". In the next steps, you use the gcloud CLI to write a log entry with unstructured data and a log entry with structured data:

  1. Write a log entry with unstructured data to the log my-test-log:

    gcloud logging write my-test-log "A simple entry."
    

    After the command completes, you see the message: Created log entry.

  2. Write a log entry with structured data to the log my-test-log:

    gcloud logging write --payload-type=json my-test-log '{ "message": "My second entry", "weather": "partly cloudy"}'
    

    When you write a log entry with structured data, you must include --payload-type=json. If you omit this field, then Logging interprets the payload as unstructured data.

If the log my-test-log doesn't exist, then Logging creates the log when the log entry is received.

List log entries by using the gcloud CLI

You can retrieve log entries from Logging and display them by using the gcloud CLI. For example, to retrieve and display the log entries with a resource type of global, run the following command:

gcloud logging read "resource.type=global"

The command returns a result similar to the following:

---
insertId: jpj9zjf73t1mn
jsonPayload:
  message: My second entry
  weather: partly cloudy
logName: projects/myloggingproject/logs/my-test-log
receiveTimestamp: '2018-11-01T18:39:31.114507977Z'
resource:
  labels:
    project_id: myloggingproject
  type: global
timestamp: '2018-11-01T18:39:31.114507977Z'
---
insertId: vd4m1if7h7u1a
logName: projects/myloggingproject/logs/my-test-log
receiveTimestamp: '2018-11-01T18:39:19.718100792Z'
resource:
  labels:
    project_id: myloggingproject
  type: global
textPayload: A simple entry
timestamp: '2018-11-01T18:39:19.718100792Z'

List log entries by using APIs Explorer

To run Logging API methods without writing any code, see Using the APIs Explorer. To read a list of log entries from Logging, do the following:

  1. Go to the API reference page for the entries.list API method:

    Go to entries.list API page

  2. Configure and run the API command:

    1. Replace PROJECT_ID in the following text:

      "resourceNames": [
      "projects/PROJECT_ID"
      ],
      "filter": "resource.type=global",
      "orderBy": "timestamp desc"
      
    2. Copy the updated text from the previous step, and paste it into the Request body field of APIs Explorer.

    3. Click Execute.

    The method returns a response similar to the following:

    {
      "entries": [
        {
          "textPayload": "A simple entry",
          "insertId": "vd4m1if7h7u1a",
          "resource": {
            "type": "global",
            "labels": {
              "project_id": "myloggingproject"
            }
          },
          "timestamp": "2018-11-01T18:39:19.718100792Z",
          "logName": "projects/myloggingproject/logs/my-test-log",
          "receiveTimestamp": "2018-11-01T18:39:19.718100792Z"
        },
        {
          "insertId": "jpj9zjf73t1mn",
          "jsonPayload": {
            "message": "My second entry",
            "weather": "partly cloudy"
          },
          "resource": {
            "type": "global",
            "labels": {
              "project_id": "myloggingproject"
            }
          },
          "timestamp": "2018-11-01T18:39:31.114507977Z",
          "logName": "projects/myloggingproject/logs/my-test-log",
          "receiveTimestamp": "2018-11-01T18:39:31.114507977Z"
        }
      ]
    }
    

View log entries in the Logs Explorer

To view log entries in the Google Cloud console, you can use the Logs Explorer. Most Google Cloud projects store a large number of logs; you can select certain log entries by writing a query.

To view the log entries that you wrote using the Logs Explorer, do the following:

  1. In the navigation panel of the Google Cloud console, select Logging, and then select Logs Explorer:

    Go to Logs Explorer

    Ensure your Google Cloud project is selected in the Google Cloud navigation bar. If necessary, use the Google Cloud project drop-down list to select your Google Cloud project.

  2. In the Resource menu, select Global.

    If you don't see the Global menu option or if you don't see your log entries, then wait a few minutes and refresh the page. It can take a few minutes for Logging to receive log entries.

  3. To view the details of a log entry, click its Menu.

    The first log entry has its data stored in textPayload. The second log entry contains structured data that is stored in jsonPayload. The structured payload contains the keys message and weather.

For information about the data format of log entries, see the LogEntry type.

Query log entries in the Logs Explorer

You can query log entries by using the query editor and, with structured logs, by the key and value. For example, to display all log entries that contain the text simple, do the following:

  1. In the navigation panel of the Google Cloud console, select Logging, and then select Logs Explorer:

    Go to Logs Explorer

  2. In the Resource menu, select Global.

  3. In the query editor, enter the string simple in quotation marks. The logs display shows only the log entry A simple entry.

  4. After you have viewed your log, remove the query string you added and click Run query. Both log entries reappear in the display.

To display all log entries with structured data that have a key of weather where the value field contains partly, do the following:

  1. The query editor contains the line resource.type="global". Enter the following command:

    jsonPayload.weather:partly
    
  2. Click Run query. The result is the single log entry My second entry.

Logs Explorer also offers saved, suggested, and recent queries. For more information about queries, see Build queries in the Logs Explorer.

For sample queries, see Sample queries using the Logs Explorer.

Troubleshooting

  • Typographical errors and unknown field names result in the gcloud CLI commands completing with invalid argument messages. For example, if you forget the period in resource.type, then it results in the error:

     ERROR: (gcloud.logging.read) INVALID_ARGUMENT: Field not found: 'resourcetype'.
    
  • When Cloud Logging hasn't been granted the necessary access permissions, the gcloud CLI commands complete with permission denied messages. For example, if a Compute Engine VM instance is configured with the default API settings, then the list command completes with a permission denied error:

     ERROR: (gcloud.logging.read) PERMISSION_DENIED: Request had insufficient authentication scopes.
    

    To fix this condition, modify your Compute Engine VM instance permissions to grant Cloud Logging read permission by doing the following:

    1. Go to the VM instance details page for your VM instance. Click Stop. This action might take a minute or two to complete.
    2. To modify the configuration, click Edit.
    3. Search for the header Cloud API access scopes, and click Details to display the settings for each API. Change the entry from Cloud Logging API to Full. Click Save.
    4. To restart your VM instance, click Start. After a few moments, your VM is ready to use.
  • When APIs Explorer can't complete your command, or requires additional authorization, it displays a message or error code:

    • 200 response code and no entries: If the message nextPageToken is displayed, then it indicates that APIs Explorer didn't have time to complete the search. Add a pageToken to your request, set the value to be the same as that given with the key nextPageToken, and then retry the command.
    • 400 response code: The query value is invalid. For example, if you misspell global as gloobal, then the message is Unsupported resource type: gloobal.
    • 404 response code: The project ID is invalid. Check the spelling of your project identifier.
    • You might be asked to sign into your Google account and permit APIs Explorer to access your account.

Clean up

To avoid incurring charges to your Google Cloud account for the resources used on this page, follow these steps.

  1. (Optional) To delete the log entries you created, run the following gcloud command:

    gcloud logging logs delete my-test-log
    

    If you don't delete your log entries, then they expire and are removed. For retention information, see Quotas & limits.

What's next