Configure transfer logs

This page describes how to configure Cloud Logging for Storage Transfer Service logs. For information on reading transfer logs, see Read transfer logs.

For information on Transfer service for on-premises data logs, see Transfer for on-premises log format.

Before you begin

Before you begin, verify that you have access to Cloud Logging. We recommend the Logs Viewer (roles/logging.viewer) Identity and Access Management role. For more information on Logging access, see Access control guide.

The following describe how to verify and grant IAM access:

Create a logging configuration

A logging configuration describes the transfer actions and the transfer action states that you're interested in logging. For example, the actions you can log are finding, deleting, and copying files. The action states you can log are success and failure. You create a logging configuration for each transfer job that you're interested in logging.

To create a logging configuration, use transferJobs.create with a LoggingConfig:

{
  "name":"transferJobs/myFirstTransfer",
  "description": "My First Transfer",
  "status": "ENABLED",
  "projectId": "test-id-001",
  "schedule": {
      "scheduleStartDate": {
          "year": 2021,
          "month": 10,
          "day": 20
      },
      "scheduleEndDate": {
          "year": 2024,
          "month": 10,
          "day": 22
      },
  },
  "loggingConfig": {
     "logActions": ["FIND", "DELETE", "COPY"],
     "logActionStates": ["SUCCEEDED", "FAILED"],
  },
  "transferSpec": {
      "awsS3DataSource": {
          "bucketName": "AWS_SOURCE_NAME",
          "awsAccessKey": {
              "accessKeyId": "AWS_ACCESS_KEY_ID",
              "secretAccessKey": "AWS_SECRET_ACCESS_KEY"
          }
      },
      "gcsDataSink": {
           "bucketName": "destination_bucket"
           "path": "foo/bar/"
      },
   }
}

You can adjust loggingConfig, to include specific logActions that you're interested in. For example, if you want to log when copy and find actions fail, you can provide the following loggingConfig:

"loggingConfig": {
  "logActions": ["COPY", "FIND"],
  "logActionStates": ["FAILED"],
}

Update a logging configuration

To update an existing transfer's logging configuration, use transferJobs.patch with LoggingConfig:

{
  "projectId: "test-id-001",
  transferJob: {
    "loggingConfig": {
       "logActions": ["FIND", "DELETE", "COPY"],
       "logActionStates": ["SUCCEEDED", "FAILED"],
    },
  },
  "updateTransferJobFieldMask": "loggingConfig"
}

What's next