logging - Configure or retrieve logging on buckets

Synopsis

gsutil logging set on -b <logging_bucket_name> [-o <log_object_prefix>] gs://<bucket_name>...
gsutil logging set off gs://<bucket_name>...
gsutil logging get gs://<bucket_name>

Description

Cloud Storage offers usage logs and storage logs in the form of CSV files that you can download and view. Usage logs provide information for all of the requests made on a specified bucket and are created hourly. Storage logs provide information about the storage consumption of that bucket for the last day and are created daily.

Once set up, usage logs and storage logs are automatically created as new objects in a bucket that you specify. Usage logs and storage logs are subject to the same pricing as other objects stored in Cloud Storage.

The logging command has two sub-commands:

Set

The set sub-command has two sub-commands:

On

The "gsutil logging set on" command will enable usage logging of the buckets named by the specified URLs, outputting log files in the specified logging_bucket. Cloud Storage doesn't validate the existence of logging_bucket so users should ensure it already exists, and all URLs must name buckets (e.g., gs://bucket). The required bucket parameter specifies the bucket to which the logs are written, and the optional log_object_prefix parameter specifies the prefix for log object names. The default prefix is the bucket name. For example, the command:

gsutil logging set on -b gs://my_logging_bucket -o UsageLog \
    gs://my_bucket1 gs://my_bucket2

will cause all read and write activity to objects in gs://mybucket1 and gs://mybucket2 to be logged to objects prefixed with the name "UsageLog", with those log objects written to the bucket gs://my_logging_bucket.

In addition to enabling logging on your bucket(s), you will also need to grant [e-mail] write access to the log bucket, using this command:

gsutil acl ch -g [e-mail]:W gs://my_logging_bucket

Note that log data may contain sensitive information, so you should make sure to set an appropriate default bucket ACL to protect that data. (See gsutil help defacl.)

Off

This command will disable usage logging of the buckets named by the specified URLs. All URLs must name buckets (e.g., gs://bucket).

No logging data is removed from the log buckets when you disable logging, but Cloud Storage will stop delivering new logs once you have run this command.

Get

If logging is enabled for the specified bucket url, the server responds with a JSON document that looks something like this:

{
  "logBucket": "my_logging_bucket",
  "logObjectPrefix": "UsageLog"
}

You can download log data from your log bucket using the gsutil cp command.

Usage Log And Storage Data Fields

For a complete list of usage log fields and storage data fields, see: https://cloud.google.com/storage/docs/access-logs#format