Create and manage log scopes

Preview

This product or feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of the Service Specific Terms. Pre-GA products and features are available "as is" and might have limited support. For more information, see the launch stage descriptions.

This document describes how you can create and manage log scopes, which you can use to help you efficiently find the log entries that you want to view or analyze. If you only want to view and analyze the log entries that originate in a project, folder, or organization, then this document isn't for you. However, if you rely on log sinks to route logs to other projects or to user-defined log buckets, or if you use log views, then the information in this document might help you efficiently find specific log entries.

This document doesn't describe how to view your logs. For information about that topic, see View logs by using the Logs Explorer.

About log scopes

Log scopes are persistent, project-level resources that list a set of resources. These resources can be projects, folders, organizations, and log views. For example, you could define a log scope that lists the projects that contain resources used for production, or one that lists the log views that include log entries for a specific resource type.

When you create a Google Cloud project, folder, or organization resource, Logging creates a log scope named _Default. This scope includes the project, folder, or organization that was created. When a searched resource is a Google Cloud project, folder, or organization, the results include the log entries that originate in the resource and then are stored in a log bucket. When a project is searched, the results also include log entries that are routed to the project by a sink in another project and then stored in a log bucket.

You can create log scopes. You can also edit and delete the log scopes that you create. However, you can't edit or delete the log scope named _Default.

You use a log scope to control which resources the Logs Explorer page searches for log data. When you open that page and select a log scope, the page searches the resources listed in that scope and then refreshes the display.

You can also use a log scope to control which resources a logs panel searches for log data. A logs panel is a custom-dashboard widget that displays log data. Each logs panel has its own configuration, which lets you create a dashboard that contains multiple logs panels where each panel displays different log data. For more information, see Display logs and errors on a custom dashboard.

For projects, the default log scope determines the set of resources that the Logs Explorer page searches when it opens. However, your Identity and Access Management (IAM) roles on the searched resources and the time-range setting determine which log entries are fetched from storage. When projects are created, the log scope named _Default is designated as the default log scope. You can set which log scope is the default log scope.

Best practices

Because log scopes provide a way for you to define and save a configuration for future use, we recommend that you create log scopes for complex search configurations.

For example, suppose that you are troubleshooting an issue and want to view the log entries for all virtual machine (VM) instances owned by your team. To accomplish this task, you might do the following:

1. You determine that the log entries that you want to view are stored in multiple log buckets and in multiple projects. For most log buckets, a log view exists that includes the log entries that you want to analyze. Where a log view doesn't exist, you can create one.

2. You decide to create a log scope because you expect to have a similar troubleshooting task in the future.

3. You open the Logs Explorer page in the Google Cloud console and then use the Refine scope menu to select your new log scope.

4. You review the log entries and find the information you need to resolve the issue you were investigating.

5. After you resolve the issue, you share the failure cause with your colleagues. You also share that you expect to see similar failures in the future, so you created a log scope that will let you, or whomever is investigating the failure, quickly find relevant log entries.

Limitations

• You can't delete or modify the log scope named _Default.
• Only Google Cloud projects support a default log scope.
• You can't add folders or organizations to a user-defined log scope.
• Log scopes are created in the global location.

Before you begin

• To get the permissions that you need to create and view log scopes, and to set the default log scope, ask your administrator to grant you the following IAM roles:

  • To create and view log scopes or to get the default log scope: Logs Configuration Writer (roles/logging.configWriter) on your project, folder, or organization
  • To set the default log scope: Observability Editor (roles/observability.editor) on your project, folder, or organization

  For more information about granting roles, see Manage access to projects, folders, and organizations.

  These predefined roles contain the permissions required to create and view log scopes, and to set the default log scope. To see the exact permissions that are required, expand the Required permissions section:

  Required permissions

  The following permissions are required to create and view log scopes, and to set the default log scope:

  • To get and set the default log scope: observability.scopes.{get, update}

  You might also be able to get these permissions with custom roles or other predefined roles.

• Select the tab for how you plan to use the samples on this page:

  Console

  When you use the Google Cloud console to access Google Cloud services and APIs, you don't need to set up authentication.

  gcloud

  In the Google Cloud console, activate Cloud Shell.

  Activate Cloud Shell

  At the bottom of the Google Cloud console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.

  Terraform

  To use the Terraform samples on this page in a local development environment, install and initialize the gcloud CLI, and then set up Application Default Credentials with your user credentials.

  1. Install the Google Cloud CLI.

  2. To initialize the gcloud CLI, run the following command:

     gcloud init

  3. If you're using a local shell, then create local authentication credentials for your user account:

     gcloud auth application-default login

     You don't need to do this if you're using Cloud Shell.

  For more information, see Set up authentication for a local development environment in the Google Cloud authentication documentation.

  REST

  To use the REST API samples on this page in a local development environment, you use the credentials you provide to the gcloud CLI.

  Install the Google Cloud CLI, then initialize it by running the following command:

  gcloud init

  For more information, see Authenticate for using REST in the Google Cloud authentication documentation.

List log scopes

Console

To list the log scopes, do the following:

1. In the Google Cloud console, go to the  Settings page:

   Go to Settings

   If you use the search bar to find this page, then select the result whose subheading is Monitoring.

2. Select the Log Scopes tab.

   The table lists your log scopes. When you've selected a Google Cloud project, one entry in the table is shown with a "Default" icon, , which indicates that this log scope lists the resources that the Logs Explorer searches when that page is opened.

gcloud

Note: The Google Cloud CLI commands on this page are configured to query a Google Cloud project. To query a folder, replace the --project field with the --folder field. Similarly, to query an organization, replace the --project field with the --organization field.

To list the log scopes in a project, use the gcloud logging scopes list command:

 gcloud logging scopes list --project=PROJECT_ID

Before running the command, update the following fields:

• PROJECT_ID: The identifier of the project.

To get the details of a log scope in a project, use the gcloud logging scopes describe command:

 gcloud logging scopes describe LOG_SCOPE --project=PROJECT_ID

Before running the command, update the following fields:

• PROJECT_ID: The identifier of the project.
• LOG_SCOPE: The name of the log scope. For example, my-scope.

Terraform

You can use Terraform to create and modify a log scope. However, you can't use Terraform to list log scopes.

REST

The Cloud Logging API contains commands that list the log scopes in a resource, or that report the details of a specific log scope. For a complete list of commands, see the API reference documentation.

For Google Cloud projects, use the following commands:

• projects.locations.logScopes.list.
• projects.locations.logScopes.get.

In the API command, set the locations field to global.

Set the default log scope

The default log scope lists the resources searched by the Logs Explorer page when that page opens. If a default log scope doesn't exist or isn't accessible, then that page automatically searches for log entries that originate in the selected project, folder, or organization. The log entries displayed by the Logs Explorer page depend on the searched resources, the time-range setting, and your IAM roles on the searched resources.

When projects are created, the log scope named _Default is created and is designated as the default log scope. However, you can create your own log scope and designate it as the default log scope.

Console

To set the default log scope, do the following:

1. In the Google Cloud console, go to the  Settings page:

   Go to Settings

   If you use the search bar to find this page, then select the result whose subheading is Monitoring.

2. Select the Log Scopes tab.

3. Find the log scope that you want to designate as the default log scope, click  More, and then select Set as default.

   The log scope you selected is shown with a "Default" icon, .

gcloud

Not supported.

Terraform

You can use Terraform to create and modify a log scope. However, you can't use Terraform to set the default log scope.

REST

Not supported.

Create a log scope

You can create 100 log scopes per project. A log scope can include a total of 100 log views and projects; however, it can only include 5 projects. You can't add folders or organizations to a log scope.

Console

To create a log scope, do the following:

1. In the Google Cloud console, go to the  Settings page:

   Go to Settings

   If you use the search bar to find this page, then select the result whose subheading is Monitoring.

2. Select the Log Scopes tab and then click Create log scope.

3. To add one or more projects, click Add projects, and complete the dialog.

   After you add a project, your IAM roles on that project determine which log entries that you can view. For example, your IAM role might let you view only those log entries that are accessible by a specific log view on a log bucket. For more information about roles, see Logging roles.

4. To add one or more log views, click Add log views, and complete the dialog.

   The dialog lists all log views that have log entries that originate in the current project, or that were routed to the current project by a sink in another project. For example, if you haven't configured any sinks, then this dialog lists the log views in your current project.

   To list log views stored in another Google Cloud project, click  Import project, and then select the Google Cloud project.

   After you add a log view, your IAM roles on either the log view or the project that stores the log view determine which log entries that you can access. For more information, see Control access to a log view.

5. In the Name log scope section, enter the name and description that you want displayed on the Log Scopes tab.

   The name of a log scope can't be modified and it must be unique within the project.

6. Click Create log scope.

gcloud

Note: The Google Cloud CLI commands on this page are configured to query a Google Cloud project. To query a folder, replace the --project field with the --folder field. Similarly, to query an organization, replace the --project field with the --organization field.

To create a log scope in a project, use the gcloud logging scopes create command:

 gcloud logging scopes create LOG_SCOPE --project=PROJECT_ID \
   --description=DESCRIPTION \
   --resource-names=RESOURCE_NAMES

Before running the command, update the following fields:

• PROJECT_ID: The identifier of the project.

• LOG_SCOPE: The name of the log scope. For example, my-scope.

• DESCRIPTION: Optional. The description of the log scope. Format the description as a string.

• RESOURCE_NAMES: A comma-separated list of the fully-qualified names of projects or log views. For example, to include my-project in the log scope, specify projects/my-project.

Terraform

To learn how to apply or remove a Terraform configuration, see Basic Terraform commands. For more information, see the Terraform provider reference documentation.

To create a log scope in a project, folder, or organization by using Terraform, use the Terraform resource google_logging_log_scope.

In the command, set the following fields:

• parent: The fully-qualified name of your project, folder, or organization. For example, you might set this field to "projects/PROJECT_ID", where PROJECT_ID is the ID of your Google Cloud project.
• locations: Set to "global".

• name: Set to the fully-qualified name of the log scope. For projects, the format of this field is:

  "projects/PROJECT_ID/locations/global/logScopes/LOG_SCOPE"
  

  In the previous expression, LOG_SCOPE is the name of a log scope, such as "production".

• resource_names: A array of projects and log views, where each project and log view is specified by using their fully-qualified name.

• description: A brief description. For example, "Scope for production resources".

REST

The Cloud Logging API also supports creating log scopes in a folder or organization. For more information, see the API reference documentation.

For Google Cloud projects, use the following command:

• projects.locations.logScopes.create.

In the API command, set the locations field to global.

Modify or delete a log scope

Console

To modify or delete a log scope that you or a colleague created, do the following:

1. In the Google Cloud console, go to the  Settings page:

   Go to Settings

   If you use the search bar to find this page, then select the result whose subheading is Monitoring.

2. Select the Log Scopes tab.

3. Find the Log Scopes that you want to modify or delete, click  More, and then do one of the following:

   • To modify, select Edit scope, and then complete the dialog.
   • To delete, select Delete scope, and then complete the dialog.

gcloud

Note: The Google Cloud CLI commands on this page are configured to query a Google Cloud project. To query a folder, replace the --project field with the --folder field. Similarly, to query an organization, replace the --project field with the --organization field.

To modify the description of list of resources in a log scope in a project, use the gcloud logging scopes update command:

 gcloud logging scopes update LOG_SCOPE --project=PROJECT_ID \
   --description=DESCRIPTION \
   --resource-names=RESOURCE_NAMES

Before running the command, update the following fields:

• PROJECT_ID: The identifier of the project.

• LOG_SCOPE: The name of the log scope. For example, my-scope.

• DESCRIPTION: The description of the log scope. Format the description as a string. Omit this field when you don't want to change the description of the log scope.

• RESOURCE_NAMES: A comma-separated list of the fully-qualified names of projects or log views. Omit this field when you don't want to change the list of resources.

To delete a log scope in a project, use the gcloud logging scopes delete command:

 gcloud logging scopes delete LOG_SCOPE --project=PROJECT_ID

Before running the command, update the following fields:

• PROJECT_ID: The identifier of the project.
• LOG_SCOPE: The name of the log scope. For example, my-scope.

Terraform

To learn how to apply or remove a Terraform configuration, see Basic Terraform commands. For more information, see the Terraform provider reference documentation.

To modify a log scope in a project, folder, or organization by using Terraform, use the Terraform resource google_logging_log_scope.

REST

The Cloud Logging API contains commands that can modify or delete a log scope. For a complete list of commands, see the API reference documentation.

For Google Cloud projects, use the following commands:

• projects.locations.logScopes.patch.
• projects.locations.logScopes.delete.

In the API command, set the locations field to global.

What's next

• View logs by using the Logs Explorer
• Configure log views on a log bucket