Quickstart using Logging tools

This quickstart introduces you to some of the capabilities of Cloud Logging and shows you how to:

  • Write log entries by using the gcloud command-line tool.
  • List log entries by using the gcloud command-line tool.
  • List log entries by using the Logging API.
  • View and query log entries by using the Logs Explorer.

Before you begin

You must have a Google Cloud project with billing enabled to complete this quickstart. If you don't have a Cloud project, or if you don't have billing enabled for your Cloud project, do the following:
  1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.
  2. In the Google Cloud Console, on the project selector page, select or create a Google Cloud project.

    Go to project selector

  3. Make sure that billing is enabled for your Cloud project. Learn how to confirm that billing is enabled for your project.

  4. In the Google Cloud Console, on the project selector page, select or create a Google Cloud project.

    Go to project selector

  5. Make sure that billing is enabled for your Cloud project. Learn how to confirm that billing is enabled for your project.

Getting started

The Cloud SDK has a group of commands, gcloud logging, that provide a command-line interface to the Cloud Logging API.

You can use the Cloud Shell environment or a Compute Engine virtual machine (VM) instance for the gcloud command-line tool commands in this quickstart. The Cloud SDK is pre-installed in the Cloud Shell environment.

Cloud Shell

Verify the Cloud SDK is configured to use the correct Cloud project:

  1. From the Cloud Console, click Activate Cloud Shell:

    Screenshot of Cloud Shell button in Cloud Console.

    The Cloud Shell opens in a window and displays a welcome message. The welcome message echoes the configured project ID:

    Screenshot of Cloud Shell displaying a welcome message.

  2. If you want to use a different Cloud project than the one listed in the welcome message, run the following command after replacing PROJECT_ID with your project ID:

        gcloud config set project PROJECT_ID
    

    To get the project ID, see Creating and managing projects.

VM instance

To create a Compute Engine VM instance in the Cloud Console, follow these steps:

  1. From the Cloud Console, select Compute Engine, and then select VM instances.
  2. Select Create instance.

  3. In Identity and API access, select Set access for each API for Access scopes.

  4. Scroll through the APIs until you find the Stackdriver Logging API. Toggle the access to Full.

  5. Leave all other settings at their default values and click Create. Your VM instance is ready to use.

  6. Click SSH to connect to your VM instance shell. After a moment, a Debian GNU/Linux shell opens in a window and displays a welcome message.

  7. Run the following command to verify that your Cloud SDK, which is pre-installed, is configured for your Compute Engine project:

    gcloud config list
    
  8. If the listed Cloud project isn't the one that you want to use, then run the following command and replace PROJECT_ID with your project ID:

       gcloud config set project PROJECT_ID
    

    To get the project ID, see Creating and managing projects.

Write log entries by using the gcloud tool

Logging supports log entries with structured and unstructured data. Structured data consists of a JSON data structure; for example {"weather": "partly cloudy"}. Unstructured data is a string of characters; for example, "A simple entry". In the next steps, you use the gcloud tool to write a log entry with unstructured data and a log entry with structured data:

  1. Write a log entry with unstructured data to the log my-test-log:

    gcloud logging write my-test-log "A simple entry."
    

    After the command completes, you see the message: Created log entry.

  2. Write a log entry with structured data to the log my-test-log:

    gcloud logging write --payload-type=json my-test-log '{ "message": "My second entry", "weather": "partly cloudy"}'
    

    When you write a log entry with structured data, you must include --payload-type=json. If you omit this field, Logging interprets the payload as unstructured data.

If the log my-test-log doesn't exist, then Logging creates the log when the log entry is received.

List log entries by using the gcloud tool

You can retrieve log entries from Logging and display them by using the gcloud tool. For example, to retrieve and display the log entries with a resource type of global, run the following command:

gcloud logging read "resource.type=global"

The command returns a result similar to the following:

---
insertId: jpj9zjf73t1mn
jsonPayload:
  message: My second entry
  weather: partly cloudy
logName: projects/myloggingproject/logs/my-test-log
receiveTimestamp: '2018-11-01T18:39:31.114507977Z'
resource:
  labels:
    project_id: myloggingproject
  type: global
timestamp: '2018-11-01T18:39:31.114507977Z'
---
insertId: vd4m1if7h7u1a
logName: projects/myloggingproject/logs/my-test-log
receiveTimestamp: '2018-11-01T18:39:19.718100792Z'
resource:
  labels:
    project_id: myloggingproject
  type: global
textPayload: A simple entry
timestamp: '2018-11-01T18:39:19.718100792Z'

List log entries by using APIs Explorer

To run Logging API methods without writing any code, use APIs Explorer. To read a list of log entries from Logging, do the following:

  1. Go to the API reference page for the entries.list API method:

    Go to entries.list API page

  2. Configure and run the API command:

    1. Replace the PROJECT_ID in the following text:

        "resourceNames": [
          "projects/PROJECT_ID"
        ],
        "filter": "resource.type=global",
        "orderBy": "timestamp desc"
      
    2. Copy the updated text from the previous step, and paste it into the Request body field of APIs Explorer.

    3. Click Execute.

    The method returns a response similar to the following:

    {
      "entries": [
        {
          "textPayload": "A simple entry",
          "insertId": "vd4m1if7h7u1a",
          "resource": {
            "type": "global",
            "labels": {
              "project_id": "myloggingproject"
            }
          },
          "timestamp": "2018-11-01T18:39:19.718100792Z",
          "logName": "projects/myloggingproject/logs/my-test-log",
          "receiveTimestamp": "2018-11-01T18:39:19.718100792Z"
        },
        {
          "insertId": "jpj9zjf73t1mn",
          "jsonPayload": {
            "message": "My second entry",
            "weather": "partly cloudy"
          },
          "resource": {
            "type": "global",
            "labels": {
              "project_id": "myloggingproject"
            }
          },
          "timestamp": "2018-11-01T18:39:31.114507977Z",
          "logName": "projects/myloggingproject/logs/my-test-log",
          "receiveTimestamp": "2018-11-01T18:39:31.114507977Z"
        }
      ]
    }
    

View log entries in the Logs Explorer

To view log entries, you can use the Logs Explorer. As most Cloud projects store a large number of logs, you can view certain log entries by adding filters or writing a query.

To view the log entries that you wrote using the Logs Explorer, do the following:

  1. Go to the Logs Explorer in the Google Cloud Console:

    Go to Logs Explorer

    Ensure your Cloud project is selected in the Google Cloud navigation bar. If necessary, use the Cloud project drop-down list to select your Cloud project.

  2. In the Resource menu, select Global.

    If you don't see the Global menu option or if you don't see your log entries, then wait a few minutes and refresh the page. It can take a few minutes for Logging to receive log entries.

  3. To view the details of a log entry, click its menu .

    The first log entry has its data stored in textPayload. The second log entry contains structured data that is stored in jsonPayload. The structured payload contains the keys message and weather.

For information about log entry data formats, see the LogEntry type.

Query log entries in the Logs Explorer

You can query log entries by using a text field, and with structured logs, by the key and value. For example, to display all log entries that contain the text simple, do the following:

  1. Go to the Logs Explorer in the Google Cloud Console:

    Go to Logs Explorer

  2. In the Resource menu, select Global.

  3. In the search-query box, enter the string simple. The logs display shows only the log entry A simple entry.

  4. After you have viewed your log, remove the query string you added and click refresh (). Both log entries reappear in the display.

To display all log entries with structured data that have a key of weather where the value field contains partly, do the following:

  1. Switch to the advanced query mode by clicking the drop-down menu in the query box and then selecting Convert to advanced filter.
  2. The query box contains the line resource.type="global". Enter the following command:

    jsonPayload.weather:partly
    
  3. Click Submit filter. The result is the single log entry My second entry.

Logs Explorer also offers saved, suggested, and recent queries. For more information about queries, see Build queries in the Logs Explorer.

For sample queries, see Sample queries using the Logs Explorer.

Troubleshooting

  • Typographical errors and unknown field names result in the gcloud tool commands completing with invalid argument messages. For example, if you forget the period in resource.type, it results in the error:

     ERROR: (gcloud.logging.read) INVALID_ARGUMENT: Field not found: 'resourcetype'.
    
  • When Cloud Logging hasn't been granted the necessary access permissions, the gcloud tool commands complete with permission denied messages. For example, if a Compute Engine VM instance is configured with the default API settings, the list command completes with a permission denied error:

     ERROR: (gcloud.logging.read) PERMISSION_DENIED: Request had insufficient authentication scopes.
    

    To resolve this condition, modify your Compute Engine VM instance permissions to grant Cloud Logging read permission by doing the following:

    1. Go to the VM instance details page for your VM instance. Click Stop. This action might take a minute or two to complete.
    2. To modify the configuration, click Edit.
    3. Search for the header Cloud API access scopes, and click Details to display the settings for each API. Change the entry from Cloud Logging API to Full. Click Save.
    4. To restart your VM instance, click Start. After a few moments, your VM is ready to use.
  • When APIs Explorer can't complete your command, or requires additional authorization, it displays a message or error code:

    • 200 response code and no entries: If the message nextPageToken is displayed, it indicates that APIs Explorer didn't have time to complete the search. Add a pageToken to your request, set the value to be the same as that given with the key nextPageToken, and then retry the command.
    • 400 response code: The query value is invalid. For example, if you misspell global as gloobal, the message is Unsupported resource type: gloobal.
    • 404 response code: The project ID is invalid. Check the spelling of your project identifier.
    • You might be asked to sign into your Google account and permit APIs Explorer to access your account.

Clean up

To avoid incurring charges to your Google Cloud account for the resources used on this page, follow these steps.

  1. (Optional) To delete the log entries you created, run the following gcloud command:

    gcloud logging logs delete my-test-log
    

    If you don't delete your log entries, they expire and are removed. For retention information, see Quotas & limits.

What's next