Configure VPC Flow Logs

This page describes how to configure VPC Flow Logs. It assumes you are familiar with the concepts described in VPC Flow Logs and About VPC Flow Logs records.

Enable VPC Flow Logs

When you enable VPC Flow Logs, you enable logging for all VMs in a subnet. However, you can reduce the amount of information written to logging. For details on the parameters you can control, see Log sampling and processing.

To customize metadata fields or configure log filtering, use the Google Cloud CLI or API.

Enable VPC Flow Logs when you create a subnet

Console

  1. In the Google Cloud console, go to the VPC networks page.

    Go to VPC networks

  2. Click the network where you want to add a subnet.

  3. Click Add subnet.

  4. For Flow logs, select On.

  5. If you want to adjust log sampling and aggregation, click Configure logs and adjust any of the following:

    • The Aggregation interval.
    • Whether to include metadata in the final log entries. By default, Include metadata includes all fields.
    • The Sample rate. 100% means that all entries are kept.

  6. Populate other fields as appropriate.

  7. Click Add.

gcloud

  1. In the Google Cloud console, activate Cloud Shell.

    Activate Cloud Shell

    At the bottom of the Google Cloud console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.

  2. Run the following command:

    gcloud compute networks subnets create SUBNET_NAME \
   --enable-flow-logs \
   [--logging-aggregation-interval=AGGREGATION_INTERVAL] \
   [--logging-flow-sampling=SAMPLE_RATE] \
   [--logging-filter-expr=FILTER_EXPRESSION] \
   [--logging-metadata=LOGGING_METADATA] \
   [--logging-metadata-fields=METADATA_FIELDS] \
   [other flags as needed]

    Replace the following:

    • AGGREGATION_INTERVAL: the aggregation interval for flow logs in that subnet. The interval can be set to any of the following: 5-sec (default), 30-sec, 1-min, 5-min, 10-min, or 15-min.
    • SAMPLE_RATE: the flow sampling rate. Flow sampling can be set from 0.0 (no sampling) to 1.0 (all logs). Default is 0.5.
    • FILTER_EXPRESSION: an expression that defines which logs you want to keep. The expression has a limit of 2,048 characters. For details, see Log filtering.

    • LOGGING_METADATA: the metadata annotations that you want to include in the logs:

      • Use include-all to include all metadata annotations.
      • Use exclude-all to exclude all metadata annotations (default).
      • Use custom to include a custom list of metadata fields that you specify in METADATA_FIELDS.

    • METADATA_FIELDS: a comma-separated list of metadata fields you want to include in the logs. For example, src_instance,dst_instance. Can only be set if LOGGING_METADATA is set to custom.

API

Enable VPC Flow Logs when you create a new subnet.

POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/regions/REGION/subnetworks
{
  "logConfig": {
    "aggregationInterval": "AGGREGATION_INTERVAL",
    "flowSampling": SAMPLING_RATE,
    "filterExpr": EXPRESSION,
    "metadata": METADATA_SETTING,
    "metadataFields": METADATA_FIELDS,
    "enable": true
  },
  "ipCidrRange": "IP_RANGE",
  "network": "NETWORK_URL",
  "name": "SUBNET_NAME"
}

Replace the following:

  • PROJECT_ID: the ID of the project where the subnet will be created.
  • REGION: the region where the subnet will be created.
  • AGGREGATION_INTERVAL: the aggregation interval for flow logs in the subnet. The interval can be set to any of the following: INTERVAL_5_SEC, INTERVAL_30_SEC, INTERVAL_1_MIN, INTERVAL_5_MIN, INTERVAL_10_MIN, or INTERVAL_15_MIN.
  • SAMPLING_RATE: the flow sampling rate. Flow sampling can be set from 0.0 (no sampling) to 1.0 (all logs). Default is .0.5.
  • EXPRESSION: the filter expression you use to filter which logs are actually written. The expression has a limit of 2,048 characters. For details, see Log filtering.

  • METADATA_SETTING: the metadata annotations that you want to include in the logs:

    • Use INCLUDE_ALL_METADATA to include all metadata annotations.
    • Use EXCLUDE_ALL_METADATA to exclude all metadata annotations (default).
    • Use custom to include a custom list of metadata fields that you specify in METADATA_FIELDS.

  • METADATA_FIELDS: the metadata fields you want to capture when you have set metadata: CUSTOM_METADATA. This is a comma-separated list of metadata fields, such as src_instance, src_vpc.project_id.

  • IP_RANGE: the primary internal IP address range of the subnet.

  • NETWORK_URL: the VPC network URL where the subnet will be created.

  • SUBNET_NAME: a name for the subnet.

For more information, refer to the subnetworks.insert method.

Terraform

You can use a Terraform module to create a custom mode VPC network and subnets.

The following example creates three subnets as follows:

  • subnet-01 has VPC Flow Logs disabled. When you create a subnet, VPC Flow Logs are disabled unless you explicitly enable them.
  • subnet-02 has VPC Flow Logs enabled with the default flow log settings.
  • subnet-03 has VPC Flow Logs enabled with some custom settings.
module "test-vpc-module" {
  source       = "terraform-google-modules/network/google"
  version      = "~> 9.0"
  project_id   = var.project_id # Replace this with your project ID in quotes
  network_name = "my-custom-mode-network"
  mtu          = 1460

  subnets = [
    {
      subnet_name   = "subnet-01"
      subnet_ip     = "10.10.10.0/24"
      subnet_region = "us-west1"
    },
    {
      subnet_name           = "subnet-02"
      subnet_ip             = "10.10.20.0/24"
      subnet_region         = "us-west1"
      subnet_private_access = "true"
      subnet_flow_logs      = "true"
    },
    {
      subnet_name               = "subnet-03"
      subnet_ip                 = "10.10.30.0/24"
      subnet_region             = "us-west1"
      subnet_flow_logs          = "true"
      subnet_flow_logs_interval = "INTERVAL_10_MIN"
      subnet_flow_logs_sampling = 0.7
      subnet_flow_logs_metadata = "INCLUDE_ALL_METADATA"
      subnet_flow_logs_filter   = "false"
    }
  ]
}

To learn how to apply or remove a Terraform configuration, see Basic Terraform commands.

Enable VPC Flow Logs for an existing subnet

Console

  1. In the Google Cloud console, go to the VPC networks page.

    Go to VPC networks

  2. Click the subnet that you want to update.

  3. Click Edit.

  4. For Flow logs, select On.

  5. If you want to adjust log sampling and aggregation, click Configure logs and adjust any of the following:

    • The Aggregation interval.
    • Whether to include metadata in the final log entries. By default, Include metadata includes all fields.
    • The Sample rate. 100% means that all entries are kept.

  6. Click Save.

gcloud

  1. In the Google Cloud console, activate Cloud Shell.

    Activate Cloud Shell

    At the bottom of the Google Cloud console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.

  2. Run the following command:

    gcloud compute networks subnets update SUBNET_NAME \
   --enable-flow-logs \
   [--logging-aggregation-interval=AGGREGATION_INTERVAL] \
   [--logging-flow-sampling=SAMPLE_RATE] \
   [--logging-filter-expr=FILTER_EXPRESSION] \
   [--logging-metadata=LOGGING_METADATA] \
   [--logging-metadata-fields=METADATA_FIELDS] \
   [other flags as needed]

    Replace the following:

    • AGGREGATION_INTERVAL: the aggregation interval for flow logs in that subnet. The interval can be set to any of the following: 5-sec (default), 30-sec, 1-min, 5-min, 10-min, or 15-min.
    • SAMPLE_RATE: the flow sampling rate. Flow sampling can be set from 0.0 (no sampling) to 1.0 (all logs). Default is 0.5.
    • FILTER_EXPRESSION: an expression that defines which logs you want to keep. The expression has a limit of 2,048 characters. For details, see Log filtering.

    • LOGGING_METADATA: the metadata annotations that you want to include in the logs:

      • Use include-all to include all metadata annotations.
      • Use exclude-all to exclude all metadata annotations (default).
      • Use custom to include a custom list of metadata fields that you specify in METADATA_FIELDS.

    • METADATA_FIELDS: a comma-separated list of metadata fields you want to include in the logs. For example, src_instance,dst_instance. Can only be set if LOGGING_METADATA is set to custom.

API

Enable VPC Flow Logs for an existing subnet.

PATCH https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/regions/REGION/subnetworks/SUBNET_NAME
{
  "logConfig": {
    "enable": true
    ...other logging fields.
  },
  "fingerprint": "SUBNETWORK_FINGERPRINT"
}

Replace the following:

  • PROJECT_ID: the ID of the project where the subnet is located.
  • REGION: the region where the subnet is located.
  • SUBNET_NAME: the name of the existing subnet.
  • SUBNET_FINGERPRINT: the fingerprint ID for the existing subnet, which is provided when you describe a subnet.
  • For the other logging fields, see Enabling VPC Flow Logging when you create a subnet.

For more information, refer to the subnetworks.patch method.

View estimated log volume for existing subnets

The Google Cloud console provides an estimate of your log volume for existing subnets, which you can then use to estimate the cost of enabling flow logs. The estimate is based on flows captured at 5-second intervals for the subnet over the previous 7 days. Also, the size of each log depends on whether you enable metadata annotations.

Console

  1. In the Google Cloud console, go to the VPC networks page.

    Go to VPC networks

  2. Click the subnet that you want to estimate costs for.

  3. Click Edit.

  4. For Flow logs, select On.

  5. Click Configure logs.

  6. View Estimated logs generated per day to see the estimate.

  7. Click Cancel so that none of your changes are saved.

View which subnets have VPC Flow Logs enabled

You can check which subnets in a network have VPC Flow Logs enabled.

Console

  1. In the Google Cloud console, go to the VPC networks page.

    Go to VPC networks

  2. Click the VPC network where you want to view the subnets.

  3. Click the Subnets tab and view the Flow logs column to see if logging is on or off.

gcloud

  1. In the Google Cloud console, activate Cloud Shell.

    Activate Cloud Shell

    At the bottom of the Google Cloud console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.

  2. Run the following command:

    gcloud compute networks subnets list \
   --project PROJECT_ID \
   --network="NETWORK" \
   --format="csv(name,region,logConfig.enable)"

    Replace the following:

    • PROJECT_ID: the ID of the project you are querying.
    • NETWORK: the name of the network containing the subnets.

Update VPC Flow Logs parameters

You can modify log sampling parameters. See Log sampling and processing for details on the parameters you can control.

To customize metadata fields or configure log filtering, use the gcloud CLI or API.

Console

  1. In the Google Cloud console, go to the VPC networks page.

    Go to VPC networks

  2. Click the subnet that you want to update.

  3. Click Edit.

  4. If you want to adjust log sampling and aggregation, click Configure logs and adjust any of the following:

    • The Aggregation interval.
    • Whether to include metadata in the final log entries. By default, Include metadata includes all fields.
    • The Sample rate. 100% means that all entries are kept.

  5. Click Save.

gcloud

  1. In the Google Cloud console, activate Cloud Shell.

    Activate Cloud Shell

    At the bottom of the Google Cloud console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.

  2. Run the following command:

    gcloud compute networks subnets update SUBNET_NAME \
   [--logging-aggregation-interval=AGGREGATION_INTERVAL] \
   [--logging-flow-sampling=SAMPLE_RATE] \
   [--logging-filter-expr=FILTER_EXPRESSION] \
   [--logging-metadata=LOGGING_METADATA] \
   [--logging-metadata-fields=METADATA_FIELDS] \

    Replace the following:

    • AGGREGATION_INTERVAL: the aggregation interval for flow logs in that subnet. The interval can be set to any of the following: 5-sec (default), 30-sec, 1-min, 5-min, 10-min, or 15-min.
    • SAMPLE_RATE: the flow sampling rate. Flow sampling can be set from 0.0 (no sampling) to 1.0 (all logs). Default is 0.5.
    • FILTER_EXPRESSION: an expression that defines what logs you want to keep. The expression has a limit of 2,048 characters. For details, see Log filtering.

    • LOGGING_METADATA: the metadata annotations that you want to include in the logs:

      • Use include-all to include all metadata annotations.
      • Use exclude-all to exclude all metadata annotations (default).
      • Use custom to include a custom list of metadata fields that you specify in METADATA_FIELDS.

    • METADATA_FIELDS: a comma-separated list of metadata fields you want to include in the logs. For example, src_instance,dst_instance. Can only be set if LOGGING_METADATA is set to custom.

API

Modify the log sampling fields to update VPC Flow Logs behaviors.

PATCH https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/regions/REGION/subnetworks/SUBNET_NAME
{
  "logConfig": {
    ...fields to modify
  },
  "fingerprint": "SUBNETWORK_FINGERPRINT"
}

Replace the following:

  • PROJECT_ID: the ID of the project where the subnet is located.
  • REGION: the region where the subnet is located.
  • SUBNET_NAME: the name of the existing subnet.
  • SUBNET_FINGERPRINT: the fingerprint ID for the existing subnet, which is provided when you describe a subnet.
  • For the fields that you can modify, see Enabling VPC Flow Logging when you create a subnet.

For more information, refer to the subnetworks.patch method.

Disable VPC Flow Logs for a subnet

Console

  1. In the Google Cloud console, go to the VPC networks page.

    Go to VPC networks

  2. Click the subnet that you want to update.

  3. Click Edit.

  4. For Flow logs, select Off.

  5. Click Save.

gcloud

  1. In the Google Cloud console, activate Cloud Shell.

    Activate Cloud Shell

    At the bottom of the Google Cloud console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.

  2. Run the following command:

    gcloud compute networks subnets update SUBNET_NAME \
   --no-enable-flow-logs

API

Disable VPC Flow Logs on a subnet to stop collecting log records.

PATCH https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/regions/REGION/subnetworks/SUBNET_NAME
{
  "logConfig": {
    "enable": false
  },
  "fingerprint": "SUBNETWORK_FINGERPRINT"
}

Replace the following:

  • PROJECT_ID: the ID of the project where the subnet is located.
  • REGION: the region where the subnet is located.
  • SUBNET_NAME: the name of the existing subnet.
  • SUBNET_FINGERPRINT: the fingerprint ID for the existing subnet, which is provided when you describe a subnet.

For more information, refer to the subnetworks.patch method.

Troubleshooting

Flow logs appear to be disabled even though you enabled them

  • When you're configuring a proxy-only subnet for internal Application Load Balancers and you're using the gcloud compute networks subnets command to enable VPC Flow Logs, the command appears to succeed, but flow logs aren't actually enabled. The --enable-flow-logs flag doesn't take effect when you also include the --purpose=INTERNAL_HTTPS_LOAD_BALANCER flag.

    When you use the Google Cloud console or the API to enable flow logs, you see the error message: "Invalid value for field 'resource.enableFlowLogs': 'true'. Invalid field set in subnetwork with purpose INTERNAL_HTTPS_LOAD_BALANCER."

    Because proxy-only subnets have no VMs, VPC Flow Logs is not supported. This is intended behavior.

What's next