Command Line Interface

The Google Cloud SDK command, gcloud beta logging, is a command-line interface that uses Stackdriver Logging API v1 and v2. If you find that you need to use the actual API, try the APIs Explorer for Stackdriver Logging API v2.

Before you begin

  1. Go to the Stackdriver > Logging page in the Cloud Platform Console, select a project, and verify that you can see logs in your project. For more information, see the Getting Started section of the Logs Viewer.

    Go to Stackdriver > Logging

  2. Install and initialize the Cloud SDK.

  3. Install the gcloud beta commands component:

    gcloud components install beta
    
  4. Set your default project so you don't have to supply the --project flag with each command:

    gcloud config set project PROJECT_ID
    

Summary of commands

The Stackdriver Logging command-line operations are listed below. Prefix each operation with gcloud beta logging. You can get help for any gcloud command by adding --help to the command.

The following commands implement features that are present in the v2 API. The command metrics create temporarily supports only the v1 API.

Command Format Description
logs delete LOG_ID Deletes all entries from the log.
logs list Lists all log identifiers that contain any entries.
metrics create METRIC DESC FILTER v1 Creates METRIC with a description and v1 filter.
metrics delete METRIC Deletes METRIC.
metrics describe METRIC Describes METRIC (v1 or v2).
metrics list Lists all logs-based metrics (v1 and v2).
metrics update METRIC --description DESC Updates METRIC with a new description.
metrics update METRIC --filter FILTER v1 Updates METRIC with a new filter.
read FILTER v2 Reads (lists) log entries matching FILTER.
sinks create SINK DESTINATION \
--output-version-format=VERSION) \
--log-filter FILTER
v1, v2 Creates a project SINK with FILTER and DESTINATION.
sinks delete SINK Deletes a SINK associated with the project.
sinks describe SINK Describes SINK associated with the project.
sinks list --only-project-sinks Lists all project sinks (v1 and v2).
sinks list Lists all sinks (v1 and v2).
sinks update SINK DESTINATION \
--output-version-format=VERSION) \
--log-filter FILTER
v1, v2 Updates project SINK with new DESTINATION or FILTER.
write LOG MESSAGE --severity SEVERITY \
--payload-type TYPE]
Writes MESSAGE to LOG.

Deprecated commands

The following commands implement features that are only in the Stackdriver Logging API v1, which has been deprecated. Use these commands only to maintain existing log sinks and log service sinks:

Command Description
sinks create SINK DESTINATION --log LOG Creates a log SINK to DESTINATION.
sinks create SINK DESTINATION --log-service SERVICE Creates a log service SINK to DESTINATION.
sinks delete SINK --log LOG Deletes SINK associated with LOG.
sinks delete SINK --log-service SERVICE Deletes SINK associated with SERVICE.
sinks describe SINK --log LOG Describes SINK associated with LOG.
sinks describe SINK --log-service SERVICE Describes SINK associated with SERVICE.
sinks list --log LOG Lists all sinks associated with LOG.
sinks list --log-service SERVICE Lists all sinks associated with SERVICE.
sinks update SINK DESTINATION --log LOG Updates log SINK to a new DESTINATION.
sinks update SINK DESTINATION --log-service SERVICE Updates log service SINK to a new DESTINATION.

The deprecated commands are identified by the presence of the following flags:

--log=LOG_ID
Required for log sinks. Identifies a v1 log sink.
--log-service=SERVICE
Required for log service sinks. Identifies a v1 log service sink.

For more details about the deprecated commands, use the --help flag. For example:

gcloud beta logging sinks create --help

Listing logs

List all of the logs in your project that have any log entries:

gcloud beta logging logs list

Example:

gcloud beta logging logs list
NAME
apache-access
apache-error
appengine.googleapis.com/request_log
compute.googleapis.com/activity_log
pubsubtestlog
syslog

Creating logs

You create a log by writing a log entry to it. See Writing log entries on this page.

Deleting logs

Delete all entries in a log:

gcloud beta logging logs delete LOG

A log with no entries is effectively deleted and does not appear in the list of project logs. Individual log entries cannot be explicitly deleted, but they expire and are deleted according to the Stackdriver Logging retention policy. If you write new entries to the log, it will reappear. Audit logs cannot be deleted.

Example:

gcloud beta logging logs delete my-new-log
Really delete all log entries from [my-new-log]?
Do you want to continue (Y/n)?  Y
Deleted [my-new-log].

Stackdriver Error Reporting: Deleting log entries does not delete the error samples collected from logs by Stackdriver Error Reporting. To delete the samples, use the deleteEvents method of the Error Reporting API.

Writing log entries

Write a log entry. If the log does not exist, it is created. You can specify a severity for the log entry, and you can write a structured log entry by specifying --payload-type=json and writing your message as a JSON string:

gcloud beta logging write LOG STRING
gcloud beta logging write LOG JSON-STRING --payload-type=json

The write command takes the following flags:

--payload-type=(text|json)
Optional. Specifies the payload type of a new log entry.
--severity=SEVERITY
Optional. Adds a severity to a new log entry. Values are DEFAULT (default), DEBUG, INFO, NOTICE, WARNING, ERROR, CRITICAL, ALERT, or EMERGENCY. The DEFAULT value means no severity is assigned.

Example:

gcloud beta logging write my-test-log "A simple entry"

gcloud beta logging write --payload-type=json my-test-log \
    '{ "message": "My second entry", "weather": "partly cloudy"}'

To find these log entries, look in the Logs Viewer under the Global resource for the log messages:

Custom log entries

Reading log entries

Reads (lists) log entries. The filter specifies the log entries to return. The returned log entries are in the v2 log entry format, and the filter must also be in that format:

gcloud beta logging read FILTER

This command can take additional flags; use --help for information about them:

--freshness=FRESHNESS
Optional. Return entries that are not older than this value.
--order=ORDER
Optional. Ordering of returned log entries based on timestamp field. Default is newest-first.
--organization=ORGANIZATION_ID
Optional. Read log entries associated with the specified organization, rather than the default project.
--limit=LIMIT
Optional. The maximum number of log entries to list.

Example:

gcloud beta logging read "resource.type=gce_instance AND logName=projects/my-gcp-project-id/logs/syslog AND textPayload:SyncAddress" --limit 10 --format json

Following is an example of a returned log entry:

{
  "insertId": "2016-04-07|08:56:48.137651-07|10.162.32.129|-1509625619",
  "logName": "projects/my-gcp-project-id/logs/syslog",
  "resource": {
    "labels": {
      "instance_id": "15543007601548829999",
      "zone": "us-central1-a"
    },
    "type": "gce_instance"
  },
  "textPayload": "Apr  7 15:56:47 my-gce-instance google-address-manager: ERROR SyncAddresses exception: HTTP Error 503: Service Unavailable",
  "timestamp": "2016-04-07T15:56:47.000Z"
}

Exporting logs

gcloud beta logging works with both v1 and v2 export sinks. The deprecated v1 log sinks and log service sinks are not described on this page. You can get the documentation by using the --help command. For example:

gcloud beta logging sinks create --help

Once a sink is created, it exports new log entries received by Stackdriver Logging. Sinks do not export log entries received before the sink was created.

v2 sink changes

Export sinks have changed significantly from the v1 API to the v2 API:

  • Organizations. v2 sinks can be created for organizations as well as projects. A "v2 sink" or "v2 project sink" in this documentation means a "v2 project or organization sink."

  • Service accounts. Instead of using cloud-logs@google.com as the export writer's identity, v2 sinks can optionally have their own unique service accounts. These are required if the logs are in an organization or if the destination is in a different project from the logs. The service account must be added to the destination's permission list. Users are encouraged to use unique service accounts.

  • Output format. A v2 sink can output log entries in either v1 or v2 format. The format of the sink's filter must match the output format. Outputting in the v1 format makes v2 sinks backward-compatible with v1 project sinks.

  • Start and end times. v2 sinks can have start and end timestamps, which determine when the sink begins exporting and when it stops exporting. This feature is not presently accessible in the command-line tool.

  • Log and service sinks. v1 log sinks and v1 log service sinks have been deprecated. They can be implemented as v1 or v2 project sinks with appropriate filters.

For more information on v2 sinks, see LogSink.

Workflow for creating sinks

Creating a sink to export logs requires a few steps, which differ depending on whether you are using the new unique writer identities (service accounts).

Using unique writer identities. This is required if your logs are in an organization or are not in the same project as the export destination. Do the following:

  1. Create the destination Cloud Storage bucket, Cloud Pub/Sub topic, or BigQuery dataset that will receive the logs.
  2. Create the sink. Exporting begins immediately, but export attempts will fail until the following steps are completed.
  3. If needed, use the sinks describe command to fetch the sink's service account.
  4. Add the service account to the destination's permission list and give it write access. The sink's exports stop failing.

Using the cloud-logs@google.com writer identity. This procedure is suitable if your logs and the export destination are in the same project, and is the same as the procedure for v1 sinks:

  1. Create the destination Cloud Storage bucket, Cloud Pub/Sub topic, or BigQuery dataset that will receive the logs.
  2. Add group cloud-logs@google.com to the destination's permission list, giving the group write permission.
  3. Create the sink. Exporting begins immediately.

Creating sinks

This section describes how to create a sink. For information on the sink creation workflow, see Workflow for creating sinks on this page.

To export log entries in the v2 format use the following command. The FILTER must also use the v2 format:

gcloud beta logging sinks create  SINK_NAME  DESTINATION  --log-filter FILTER --output-version-format=V2

To export log entries in the v1 format, use the following command. The FILTER must also use the v1 format:

gcloud beta logging sinks create  SINK_NAME  DESTINATION  --log-filter FILTER

It is an error to export v1-format log entries and v2-format log entries to the same destination at the same time.

The following additional flags are permitted in the sinks create and sinks update commands:

--output-version-format=(V1|V2)
Optional. Specifies the output and filter version for a sink. It's value must be must be V1 (default) or V2. For more information, see LogSink.
--unique-writer-identity
Optional. Forces the sink to use a unique service account for writing logs. You can retrieve the service account name using sinks describe. For more information, see LogSink and projects.sinks.create.

Examples:

Create a v2 sink, syslog-errors, that exports syslog log entries with severity ERROR from Compute Engine VM instances:

gcloud beta logging sinks create  syslog-errors  \
    storage.googleapis.com/my-third-gcs-bucket \
    --output-version-format=V2 \
    --unique-writer-identity \
    --log-filter "resource.type=gce_instance AND logName=projects/[PROJECT_ID]/logs/compute.googleapis.com/syslog AND severity=ERROR"

Create a sink, syslog-errors-v1, that exports the same log entries in the v1 format:

gcloud beta logging sinks create  syslog-errors  \
    storage.googleapis.com/my-fourth-gcs-bucket \
    --log-filter "metadata.serviceName=compute.googleapis.com AND log=compute.googleapis.com/syslog AND metadata.severity=ERROR"

Listing sinks

List your project's sinks. With no arguments, this command lists all v1 and v2 sinks, including log sinks and log service sinks. Using the indicated flag restricts the output to only v1 and v2 project sinks:

gcloud beta logging sinks list
gcloud beta logging sinks list  --only-project-sinks

Example:

gcloud beta logging sinks list
NAME                            DESTINATION                                  TYPE          FORMAT  FILTER
google-sink-1481139614360-9906  storage.googleapis.com/my-second-gcs-bucket  LOG: syslog   V1
test-sink-1                     storage.googleapis.com/my-gcs-bucket         PROJECT SINK  V2      severity>WARNING

Describing sinks

Show the name, destination, service account, and filter assigned to a sink:

gcloud beta logging sinks describe  SINK_NAME

Example (project sink):

gcloud beta logging sinks describe my-project-sink
destination: storage.googleapis.com/my-second-gcs-bucket
filter: severity=ERROR
format: V2
name: my-project-sink
type: PROJECT SINK
writer_identity: serviceAccount:cloud-logs@system.gserviceaccount.com

Updating sinks

You can update a sink to change the destination, the filter, the output format version, or the unique writer property. The filter must use the same format as the sink's new output version:

gcloud beta logging sinks update  SINK_NAME  DESTINATION
gcloud beta logging sinks update  SINK_NAME  --log-filter FILTER
gcloud beta logging sinks update  SINK_NAME  DESTINATION  --log-filter FILTER

The following additional flags are also permitted in this command. See the sinks create command for a description of the flags.

--output-version-format=(V1|V2)
--unique-writer-identity

Notes. If you change the output version, you must change the filter. If the present sink is using a unique writer service account, you cannot revert to using cloud-logs@google.com. This command does not provide access to the start and end times of the sink, although the v2 API does.

Examples:

Update the destination of a project sink.

gcloud beta logging sinks update  my-project-sink  storage.googleapis.com/my-second-gcs-bucket

Change a sink exporting in the v1 format to use the v2 format. You must also the current v1 filter.

gcloud beta logging sinks update  my-project-sink-2 \
    --output-version-format=V2 \
    --log-filter="resource.type=gce_instance AND logName=projects/YOUR_PROJECT/logs/syslog AND robot"

Deleting sinks

Stop exporting a log, a log service, or log entries matching a filter by deleting the sink:

gcloud beta logging sinks delete SINK_NAME

Example (v1 project sink or v2 sink):

gcloud beta logging sinks delete syslog-sink-1
...

Pause exporting. Using the v2 API, you can also schedule a sink to stop exporting at a specific time. Later, you can restart exporting new log entries from that sink by changing the start and end times.

Logs-based metrics

The gcloud beta logging command presently supports creating and updating only v1-format logs-based metrics. However, except for the format of the filter, there is no difference between v1 and v2 logs-based metrics. The get, list, and describe commands will work on both v1 and v2 logs-based metrics.

Example: Create a v1 logs-based metric called error_count.

gcloud beta logging metrics create error_count \
    "Syslog error counts." \
    "metadata.serviceName=compute.googleapis.com AND log=syslog AND metadata.severity>=ERROR"

Example: List the project's logs-based metrics. Note: This command and the describe command list both v1 and v2 logs-based metrics, but presently they do not indicate the format used by the metrics:

gcloud beta logging metrics list
NAME         DESCRIPTION           FILTER
error_count  Syslog error counts.  metadata.serviceName=compute.googleapis.com AND log=syslog AND metadata.severity>=ERROR

Example: Delete the error_count metric.

gcloud beta logging metrics delete error_count

What's next

See the Stackdriver Logging API v2.

Monitor your resources on the go

Get the Google Cloud Console app to help you manage your projects.

Send feedback about...

Stackdriver Logging