Logging
Cloud NAT logging allows you to log NAT connections and errors. When Cloud NAT logging is enabled, one log entry can be generated for each of the following scenarios:
- When a network connection using NAT is created.
- When a packet is dropped because no port was available for NAT.
You can choose to log both kinds of events, or only one or the other.
Created logs are sent to Cloud Logging.
Specifications
The following specifications apply to Cloud NAT logging:
Cloud NAT logging handles TCP and UDP traffic only.
Cloud NAT logging only logs dropped packets if they are egress (outbound) TCP and UDP packets. It does not log dropped incoming packets. For example, if an inbound response to an outbound request is dropped for any reason, no error is logged.
Each VM instance can only generate a certain number of log entries per unit time, proportional to its number of vCPUs. The VM can generate 50-100 log entries per second per vCPU.
This rate threshold affects the number of events that can be logged. Even if some events are filtered out, their occurrence counts toward the number of possible log entries. Limiting logs to only errors or only network address translation connections does not necessarily increase the number of viewed log entries. For example, if you choose to log only successful connections, periods of excessive failed connection attempts and NAT errors can still restrict the number of successful connection log entries.
Cloud NAT logging does not log every single packet. Even if the VM's rate threshold has not been reached, some conditions can cause events to be omitted from the log. You should rely on the presence of entries in Cloud NAT logging to make informed decisions, but you should not assume that the absence of entries means that an event didn't happen.
Configuring logging
Use the following steps to configure Cloud NAT logging.
Enabling logging
If logging is enabled, all collected logs are sent to Cloud Logging by default. You can filter these so that only certain logs are sent.
You can also specify these values when you create or edit a NAT gateway. The following directions show how to enable logging for an existing NAT gateway.
Console
In the Google Cloud Console, go to the Cloud NAT page.
Click your NAT gateway.
Click
Edit.Click Logging, minimum ports, timeout to open that section.
Under Stackdriver logging, select one of the following:
- No logging: disables logging
- Translation and errors: sends all logs to Logging
- Translation only: sends a log only when a connection is created; does not log dropped packets
- Errors only: sends a log when a packet is dropped because no port was available; does not log new connections
Click Save.
gcloud
The following commands enable logging for an existing NAT gateway.
In each command, replace the following:
NAT_GATEWAY
: the name of the NAT gatewayROUTER_NAME
: the name of the Cloud Router that hosts the NAT gatewayREGION
: the region of the Cloud Router
To log network address translation events and errors:
gcloud compute routers nats update NAT_GATEWAY \ --router=ROUTER_NAME \ --region=REGION \ --enable-logging
To log only network address translation events:
gcloud compute routers nats update NAT_GATEWAY \ --router=ROUTER_NAME \ --region=REGION \ --enable-logging \ --log-filter=TRANSLATIONS_ONLY
To log only errors:
gcloud compute routers nats update NAT_GATEWAY \ --router=ROUTER_NAME \ --region=REGION \ --enable-logging \ --log-filter=ERRORS_ONLY
Clearing log filters
If you have a filter set, you can clear it. Clearing a log filter means that both network address translation events and errors are logged, provided that logging is enabled.
Console
In the Cloud Console, go to the Cloud NAT page.
Click your NAT gateway.
Click
Edit.Click Logging, minimum ports, timeout to open that section.
Under Stackdriver logging, select Translation and errors.
Click Save.
gcloud
gcloud compute routers nats update NAT_GATEWAY \ --router=ROUTER_NAME \ --region=REGION \ --log-filter=ALL
Replace the following:
NAT_GATEWAY
: the name of the NAT gatewayROUTER_NAME
: the name of the Cloud Router that hosts the NAT gatewayREGION
: the region of the Cloud Router--log-filter=ALL
: sets the log filter to accept all logs
Disabling logging
To disable logging, do the following:
Console
In the Cloud Console, go to the Cloud NAT page.
Click your NAT gateway.
Click
Edit.Click Logging, minimum ports, timeout to open that section.
Under Stackdriver logging, select No logging.
Click Save.
gcloud
gcloud compute routers nats update NAT_GATEWAY \ --router=ROUTER_NAME \ --region=REGION \ --no-enable-logging
Replace the following:
NAT_GATEWAY
: the name of the NAT gatewayROUTER_NAME
: the name of the Cloud Router that hosts the NAT gatewayREGION
: the region of the Cloud Router
Determining logging status
To determine the status for logging, do the following:
Console
In the Google Cloud Console, go to the Cloud NAT page.
Click your NAT gateway.
In the Logging, minimum ports, timeout section, inspect the selections under Stackdriver logging.
gcloud
gcloud compute routers nats describe NAT_GATEWAY \ --router=ROUTER_NAME \ --region=REGION
Replace the following:
NAT_GATEWAY
: the name of the NAT gatewayROUTER_NAME
: the name of the Cloud Router that hosts the NAT gatewayREGION
: the region of the Cloud Router
Viewing logs
To view NAT logs, do the following:
Console
In the Google Cloud Console, go to the Logs Viewer page.
- To see all NAT logs, in the first pull-down menu, select Cloud NAT Gateway.
- To see logs for only one region, in the first pull-down menu, select Cloud NAT Gateway, and then slide the cursor right to select a region.
- To see logs for only one gateway, in the first pull-down menu, select Cloud NAT Gateway, and then slide the cursor right to select a region. Slide the cursor right again to select a single gateway.
Alternatively, go to the Logs Viewer page and paste the following into the Filter by label or text search field:
resource.type="nat_gateway" logName="projects/{#project_id}/logs/compute.googleapis.com%2Fnat_flows"
gcloud
gcloud logging read 'resource.type=nat_gateway' \ --limit=10 \ --format=json
Where:
resource.type=nat_gateway
: limits the output to your NAT gateways--limit=10
: limits the output to 10 entries; you can input a different value to see more or fewer entries, or omit it entirely to see a continuous scroll of logs--format=json
: displays the output in JSON format
For more options, see Reading log entries.
You can configure the export of logs-based metrics for resource logs.
What is logged
Cloud NAT log entries contain information useful for monitoring and debugging your NAT traffic. Log entries contain the following types of information:
- General information shown in most Google Cloud logs, such as severity, project ID, project number, and timestamp.
- Specific information related to Cloud NAT. Some log fields contain entries that are themselves multiple fields. These entries and field descriptions are shown in the following tables.
Log fields
Field | Value | Meaning |
---|---|---|
connection |
object(NatIpConnection) | 7-tuple describing the source VM IP address and port, NAT source IP address and port, destination IP address and port, and IP address protocol of this connection. |
allocation_status |
enum | Indicates whether this connection was successfully allocated or
dropped. One of OK or DROPPED . |
gateway_identifiers |
object(NatGateway) | The NAT gateway configuration that the connection used. |
endpoint |
object(InstanceDetails) | VM instance details. In a Shared VPC configuration,
project_id corresponds to the service project. |
vpc |
object(VpcDetails) | VPC network details. In a Shared VPC configuration,
project_id corresponds to that of the host project. |
destination |
object(DestinationDetails) | Details of the destination of the connection. |
NatIpConnection field format
Field | Type | Description |
---|---|---|
src_ip |
string | Source IP address |
src_port |
int32 | Source port |
nat_ip |
string | NAT IP address |
nat_port |
int32 | NAT assigned port |
dest_ip |
string | Destination IP address |
dest_port |
int32 | Destination port |
protocol |
int32 | IANA protocol number |
NatGateway field format
Field | Type | Description |
---|---|---|
gateway_name |
string | Name of the NAT gateway |
router_name |
string | Cloud Router associated with the NAT gateway |
region |
string | Region of the Cloud Router |
InstanceDetails field format
Field | Type | Description |
---|---|---|
project_id |
string | ID of the project containing the VM |
vm_name |
string | Instance name of the VM |
region |
string | Region of the Cloud Router |
zone |
string | Zone of the VM |
VpcDetails field format
Field | Type | Description |
---|---|---|
project_id |
string | ID of the project containing the network |
vpc_name |
string | Network on which the VM is operating |
subnetwork_name |
string | Subnet on which the VM is operating |
DestinationDetails field format
Field | Type | Description |
---|---|---|
geo_location |
object(GeographicDetails) | If the destination of the connection was external to Google Cloud, this field is populated with available location metadata. |
instance |
object(InstanceDetails) | If the destination of the connection is an instance within the same project as the source, this field is populated with VM instance details. |
GeographicDetails field format
Field | Type | Description |
---|---|---|
continent |
string | Continent for external endpoints |
country |
string | Country for external endpoints |
region |
string | Region for external endpoints |
city |
string | City for external endpoints |
asn |
string | The autonomous system number (ASN) of the external network to which this endpoint belongs. |
Examples
Example 1: NAT-ed TCP connection record from a VM instance in a Shared VPC network going to an external server in France.
{ insertId: "1the8juf6vab1t" jsonPayload: { connection: { Src_ip: "10.0.0.1" Src_port: 45047 Nat_ip: "203.0.113.17" Nat_port: 34889 dest_ip : "198.51.100.142" Dest_port: 80 Protocol: "tcp" } allocation_status: "OK" Gateway_identifiers: { Gateway_name: "my-nat-1" router_name: "my-router-1" Region: "europe-west1" } Endpoint: { Project_id: "service-project-1" Vm_name: "vm-1" Region: "europe-west1" Zone: "europe-west1-b" } Vpc: { Project_id: "host-project" Vpc_name: "network-1" Subnetwork_name: "subnetwork-1" } Destination: { Geo_location: { Continent: "Europe" Country: "France" Region: "Nouvelle-Aquitaine" City: "Bordeaux" } } } logName: "projects/host-project/logs/compute.googleapis.com%2Fnat_flows" receiveTimestamp: "2018-06-28T10:46:08.123456789Z" resource: { labels: { region: "europe-west1-d" project_id: "host-project" router_id: "987654321123456" gateway_name: "my-nat-1" } type: "nat_gateway" } labels: { nat.googleapis.com/instance_name: "vm-1" nat.googleapis.com/instance_zone: "europe-west1-b" nat.googleapis.com/nat_ip: "203.0.113.17" nat.googleapis.com/network_name: "network-1" nat.googleapis.com/router_name: "my-router-1" nat.googleapis.com/subnetwork_name: "subnetwork-1" } timestamp: "2018-06-28T10:46:00.602240572Z" }
Example 2: Record for a packet dropped because there were no available ports. Sending VM was trying to reach the external IP address of another VM in the same project.
{ insertId: "1the8juf6vab1l" jsonPayload: { connection: { Src_ip: "10.0.128.1" Src_port: 45047 dest_ip : "192.0.2.87" Dest_port: 80 Protocol: "tcp" } allocation_status: "DROPPED" Gateway_identifiers: { Gateway_name: "my-nat-2" Cloud_router: "my-router-1" Region: "europe-west1" } Endpoint: { Project_id: "service-project-1" Vm_name: "vm-1" Region: "europe-west1" Zone: "europe-west1-b" } Vpc: { Project_id: "host-project" Vpc_name: "network-1" Subnetwork_name: "subnetwork-1" } Destination: { Instance: { Project_id: "service-project-1" Vm_name: "vm-2" Region: "asia-east1" Zone: "asia-east1-b" } } } logName: "projects/host-project/logs/compute.googleapis.com%2Fnat_flows" receiveTimestamp: "2018-06-28T10:46:09.123456789Z" resource: { labels: { region: "europe-west1-d" project_id: "host-project" router_id: "987654321123456" gateway_name: "my-nat-2" } type: "nat_gateway" } timestamp: "2018-06-28T10:46:01.602240572Z" }
Logging pricing
See Logging pricing.
Monitoring
Cloud NAT exposes key metrics to Cloud Monitoring that give you insights into your fleet's usage of NAT gateways.
Metrics are sent automatically to Cloud Monitoring. There, you can create custom dashboards, set up alerts, and query the metrics.
Following are the required Identity and Access Management (IAM) roles:
For Shared VPC users with VMs and NAT gateways defined in different projects, access to the VM level metrics requires the
roles/monitoring.viewer
IAM role for the project of each VM.For the NAT gateway resource, access to the gateway metrics requires the
roles/monitoring.viewer
IAM role for the project that contains the gateway.
Viewing predefined dashboards
Cloud NAT provides a set of predefined dashboards that display activity across your gateway:
- Open connections
- Egress data processed
- Ingress data processed
- Port utilization
- NAT allocation errors
To view predefined dashboards from the details page of a particular gateway, see the following steps:
Console
In the Cloud Console, go to the Cloud NAT page.
Click an existing gateway.
Click the Monitoring tab.
Scroll down to see all the dashboards. You can change the view from 1 hour to 30 days by using the control at the top of the page. Hovering over a point on the graph gives you details for that specific time.
Defining custom dashboards
To create custom Cloud Monitoring dashboards over Cloud NAT metrics, see the following steps:
Console
In the Cloud Console, go to Monitoring.
Click Dashboards, and then click Create dashboard.
Create a name for your dashboard, and then click Confirm.
Click Add chart.
Give the chart a title.
Specify the resource type and metric:
- For Cloud NAT metrics for a VM, search by using the
compute.googleapis.com/nat/
prefix. - For metrics for a NAT gateway, use the
router.googleapis.com/nat/
prefix.
For lists of metrics and filtering dimensions, see Using VM instance metrics and Using NAT gateway metrics.
- For Cloud NAT metrics for a VM, search by using the
Specify any filtering dimensions.
Click Save.
Defining alerting policies
Console
You can create alerting policies to monitor the values of metrics and to notify you when those metrics violate a condition.
To create an alerting policy that monitors one or more Cloud NAT gateway or Compute Engine VM instance resources, follow these steps:
- In the Google Cloud Console, go to the Monitoring page.
If you have never used Cloud Monitoring, then on your first access of Monitoring in the Google Cloud Console, a Workspace is automatically created and your project is associated with that Workspace. Otherwise, if your project isn't associated with a Workspace, then a dialog appears and you can either create a Workspace or add your project to an existing Workspace. We recommend that you create a Workspace. After you make your selection, click Add.
- In the Monitoring navigation pane, select notificationsAlerting, and then select Create policy.
- Click Add condition:
- The settings in the Target pane specify the resource and metric to be monitored. In the Find resource type and metric field, select the resource Cloud NAT gateway or Compute Engine VM instance. Next, select a metric from the metrics list.
- The settings in the Configuration pane of the alerting policy determine when the alert is triggered. Most fields in this pane are populated with default values. For more information about the fields in the pane, see Configuration in the Alerting policies documentation.
- Click Add.
- To advance to the notifications section, click Next.
- Optional: To add notifications to your alerting policy, click
Notification channels. In the dialog, select one or more notification
channels from the menu, and then click OK.
If a notification channel that you want to add isn't listed, then click Manage notification channels. You are taken to the Notification channels page in a new browser tab. From this page, you can update the configured notification channels. After you have completed your updates, return to the original tab, click autorenewRefresh, and then select the notification channels to add to the alerting policy.
- To advance to the documentation section, click Next.
- Click Name and enter a name for the alerting policy.
- Optional: Click Documentation, and then add any information that you want included in a notification message.
- Click Save.
Metric reporting frequency and retention
Metrics for Cloud NAT are batched and exported to Monitoring every one (1) minute. Monitoring data is retained for six (6) weeks. The dashboard provides data analysis in default intervals of 1H (one hour), 6H (six hours), 1D (one day), 1W (one week), and 6W (six weeks). You can manually request analysis in any interval from 6W to 1 minute.
Using VM instance metrics
The following metric type names must be prefixed with
compute.googleapis.com/nat/
. That prefix has been omitted from the entries
in the table.
metric.type | Type | Description |
---|---|---|
sent_bytes_count |
COUNTER | Count of bytes sent (VM -> internet) over the NAT gateway. |
sent_packets_count |
COUNTER | Count of packets sent (VM -> internet) over the NAT gateway. |
received_bytes_count |
COUNTER | Count of bytes received (internet -> VM) over the NAT gateway. |
received_packets_count |
COUNTER | Count of packets received (internet -> VM) over the NAT gateway. |
new_connections_count |
COUNTER | Count of new connections created over the NAT gateway. |
open_connections |
GAUGE | Number of connections currently open on the NAT gateway. |
closed_connections_count |
COUNTER | Count of connections closed over the NAT gateway. |
dropped_sent_packets_count |
COUNTER | Count of sent packets dropped by the NAT gateway (because of port exhaustion or because of port conflict due to endpoint independent mapping). |
dropped_received_packets_count |
COUNTER | Count of received packets dropped by the NAT gateway because a connection was not found. |
port_usage |
GAUGE | Maximum number of connections from a VM to a single internet endpoint (IP:port). |
allocated_ports |
GAUGE | Number of ports allocated to a VM by the NAT gateway. |
VM instance filtering dimensions
label_key | Type | Description |
---|---|---|
project_id |
STRING | The project ID of the VM instance. |
instance_id |
STRING | The ID of the VM instance. |
zone |
STRING | The zone of the VM instance. |
nat_project_number |
STRING | The project number to which the NAT gateway belongs. |
router_id |
STRING | The Cloud Router ID to which the NAT gateway belongs. |
nat_gateway_name |
STRING | The name of the NAT gateway. |
nat_ip |
STRING | The NAT IP address allocated to the NAT gateway. Valid for metrics
allocated_ports . |
ip_protocol |
STRING | The protocol of the connection. Can be TCP ,
UDP or ICMP . Valid for all metrics except
allocated_ports . |
reason |
STRING | The reason for the packet drop. Possible values are:
Valid for metrics |
Using NAT gateway metrics
The following metric type names must be prefixed with
router.googleapis.com/nat/
. That prefix has been omitted from the entries
in the table.
metric.type | Type | Description |
---|---|---|
nat_allocation_failed |
BOOL | Indicates `true` if there is a failure in allocating NAT IP addresses to any VM in the NAT gateway, else `false`. A common reason is having insufficient NAT IP addresses in the gateway. |
sent_bytes_count |
COUNTER | Count of bytes sent (VMs -> internet) over the NAT gateway. |
sent_packets_count |
COUNTER | Count of packets sent (VMs -> internet) over the NAT gateway. |
received_bytes_count |
COUNTER | Count of bytes received (internet -> VMs) over the NAT gateway. |
received_packets_count |
COUNTER | Count of packets received (internet -> VMs) over the NAT gateway. |
new_connections_count |
COUNTER | Count of new connections created over the NAT gateway. |
open_connections |
GAUGE | Number of connections currently open on the NAT gateway. |
closed_connections_count |
COUNTER | Count of connections closed over the NAT gateway. |
dropped_sent_packets_count |
COUNTER | Count of sent packets dropped by the NAT gateway (because of port exhaustion, or port conflict due to endpoint independent mapping). |
dropped_received_packets_count |
COUNTER | Count of received packets dropped by the NAT gateway because a connection was not found. |
port_usage |
GAUGE | The port usage for a single VM is the maximum number of connections from the VM to a single internet endpoint (IP:port). This field shows the highest port usage among all the VMs for a NAT gateway. |
allocated_ports |
GAUGE | Number of ports allocated to all VMs by the NAT gateway. |
Gateway filtering dimensions
label_key | Type | Description |
---|---|---|
project_id |
STRING | The project ID of the gateway. |
region |
STRING | The region of the gateway. |
router_id |
STRING | The Cloud Router ID to which the NAT gateway belongs. |
gateway_name |
STRING | The name of the NAT gateway. |
ip_protocol |
STRING | The protocol of the connection. Can be TCP ,
UDP or ICMP . Valid for all metrics except
allocated_ports . |
nat_ip |
STRING | The NAT IP address allocated to the NAT gateway. Valid for metrics
allocated_ports . |
reason |
STRING | The reason for the packet drop. Possible values are:
Valid for metrics |
Using the API
You can construct arbitrary queries over the Cloud NAT metrics by using the Monitoring API's v3 projects.timeSeries.list request.
API example
Getting allocated ports of a VM, with 1m aggregates, timeSeries.list parameters:
- name: projects/PROJECT_ID
- Filter:
resource.type = "gce_instance" AND metric.name="compute.googleapis.com/nat/allocated_ports" AND metric.labels.nat_gateway_name="nat-1-237227-1569344091-5"
- interval.start_time:
2019-09-24T16:58:53Z
- interval.end_time:
2019-09-24T16:58:53Z
- aggregation.alignmentPeriod:
60s
- aggregation.crossSeriesReducer:
REDUCE_SUM
- aggregation.perSeriesAligner:
ALIGN_SUM
"timeSeries": [ { "resource": { "labels": { "instance_id": "1406292833167995938", "project_id": "PROJECT_ID", "zone": "us-west1-c" }, "type": "gce_instance" }, "metric": { "labels": { "nat_ip": "70.32.157.11", "nat_gateway_name": "nat-1-237227-1569344091-5", "nat_project_number": "PROJECT_NUMBER", "router_id": "1380055590921303155" }, "type": "compute.googleapis.com/nat/allocated_ports" }, "metricKind": "GAUGE", "points": [ { "interval": { "endTime": "2019-09-24T16:58:53.699Z", "startTime": "2019-09-24T16:58:53.699Z" }, "value": { "int64Value": "64" } } ], "valueType": "INT64" },
Monitoring pricing
There is no cost for Cloud NAT monitoring.
What's next
- Learn more about Logging.
- Learn more about Monitoring.
- Create your own Cloud NAT gateway.
- Troubleshoot common issues.