Using request logging

Google Cloud Armor per-request logs for security policy name, match rule priority, associated action, and related information are logged as part of logging for HTTP(S) Load Balancing. Logging for new backend services is disabled by default, so you must enable HTTP(S) Load Balancing logging to record complete logging information for Google Cloud Armor.

Because Google Cloud Armor logs are part of the Cloud Load Balancing logs, Google Cloud Armor log generation is subject to the log sampling rate configured for HTTP(S) Load Balancing. If you reduce the sampling rate for your external HTTP(S) load balancer, your Google Cloud Armor requests logs are sampled at that reduced rate.

Using logging, you can view every request evaluated by a Google Cloud Armor security policy and the outcome or action taken. For example, to view denied requests, you can use filters such as jsonPayload.enforcedSecurityPolicy.outcome="DENY" or jsonPayload.statusDetails="denied_by_security_policy".

To enable logging, see HTTP(S) Load Balancing logging and monitoring.

In addition, you can set different logging levels to help you evaluate whether your security policies and their rules are working as intended. For complete information, see Verbose logging.

Security policy log entries

The following log entries in the Logs Viewer are for Google Cloud Armor security policy and rules logging. The entries include the following structure in jsonPayload. HTTP request details appear in the httpRequest message.

  • statusDetails (string): a textual description of the response code
    • redirected_by_security_policy: the request was redirected by a redirect rule, either GOOGLE_RECAPTCHA or EXTERNAL_302
    • throttled_by_security_policy: the request was blocked by a throttle rule
    • rate_based_banned_by_security_policy: the request was banned by a rate-based ban rule
    • denied_by_security_policy: a request was denied by the load balancer because of a Google Cloud Armor security policy
    • body_denied_by_security_policy: a request body was denied by the load balancer because of a Google Cloud Armor security policy
  • enforcedSecurityPolicy: the security policy rule that was enforced
    • name (string): the name of the security policy
    • priority (number): the priority of the matching rule in the security policy
    • configuredAction (string): the name of the configured action in the matching rule—for example, ALLOW, DENY, GOOGLE_RECAPTCHA, EXTERNAL_302, THROTTLE (for a throttle rule), RATE_BASED_BAN (for a rate-based ban rule)
    • outcome (string): the outcome of executing the configured action—for example, ACCEPT, DENY, REDIRECT, THROTTLE, RATE_BASED_BAN
    • preconfiguredExprIds (string): the IDs of all preconfigured WAF rule expressions that triggered the rule
  • previewSecurityPolicy: populated if a request matches a rule configured for preview (present only when a preview rule would have taken priority over the enforced rule)
    • name (string): the name of the security policy
    • priority (number): the priority of the matching rule in the security policy
    • configuredAction (string): the name of the configured action in the matching rule—for example, ALLOW, DENY, GOOGLE_RECAPTCHA, EXTERNAL_302, THROTTLE (for a throttle rule), RATE_BASED_BAN (for a rate-based ban rule)
    • outcome (string): the outcome of executing the configured action—for example, ACCEPT, DENY, REDIRECT, THROTTLE, RATE_BASED_BAN
    • preconfiguredExprIds (string): the IDs of all preconfigured WAF rule expressions that triggered the rule
  • enforcedEdgeSecurityPolicy (Preview): the edge security policy rule that was enforced
    • name (string): the name of the security policy
    • priority (number): the priority of the matching rule in the security policy
    • configuredAction (string): the name of the configured action in the matching rule—for example, ALLOW, DENY
    • outcome (string): the outcome of executing the configured action—for example, ACCEPT, DENY
  • previewEdgeSecurityPolicy (Preview): populated if a request matches an edge security policy rule configured for preview (present only when a preview rule would have taken priority over the enforced rule)
    • name (string): the name of the security policy
    • priority (number): the priority of the matching rule in the security policy
    • configuredAction (string): the name of the configured action in the matching rule—for example, ALLOW, DENY
    • outcome (string): the outcome of executing the configured action—for example, ACCEPT, DENY

Verbose logging

It can be difficult to tell why a preconfigured WAF rule was triggered by a particular request. Google Cloud Armor's default event logs contain the rule that was triggered, as well as the subsignature. However, you might need to identify the details from the incoming request that triggered the rule for troubleshooting, rule validation, or rule tuning purposes.

You can change the level of detail logged for a policy by using the --log-level flag. This flag can have the values NORMAL or VERBOSE:

--log-level=[NORMAL | VERBOSE]

For example:

gcloud compute security-policies update ca-policy-1 \
    --log-level=VERBOSE

We recommend that you enable verbose logging when you first create a policy, make changes to a policy, or troubleshoot a policy.

Values logged when verbose logging is enabled

When verbose logging is enabled, additional information is logged to the HTTP(S) Load Balancing request log that is sent to Cloud Logging. The following additional fields appear in the request log when verbose logging is enabled:

  • matchedFieldType (string): This is the type of field causing the match.

    • ARG_NAMES
    • ARG_VALUES
    • BODY

      • When the BODY field is in the log, it means that the entire post body matches a rule.
    • COOKIE_VALUES

    • COOKIE_NAMES

    • FILENAME

    • HEADER_VALUES

    • RAW_URI

    • REFERER

    • REQUEST_LINE

    • URI

    • USER_AGENT

    • HEADER_NAMES

    • ARGS_GET

    • X_FILENAME

    • ARG_NAME_COUNT

    • TRANSFER_ENCODING

    • REQUEST_METHOD

  • matchedFieldName (string): If this matches the value part of a key-value pair, the key value is stored in this field. Otherwise, it is empty.

  • matchedFieldValue (string): A prefix of up to 16 bytes for the part of the field that causes the match.

  • matchedFieldLength (integer): The total length of the field.

  • matchedOffset (integer): The start offset inside the field that causes the match.

  • matchedLength (integer): The length of the match.

For example, you might send this request to a project where SQL injection WAF rules are enabled:

curl http://IP_ADDR/?sql_table=abc%20pg_catalog%20xyz

The entry in the Logs Explorer would look like the following:

enforcedSecurityPolicy: {
 name: "user-staging-sec-policy"
 priority: 100
 configuredAction: "DENY"
 outcome: "DENY
 preconfiguredExprIds: [
   0: "owasp-crs-v030001-id942140-sqli"
  ]
matchedFieldType: "ARG_VALUES"
matchedFieldName: "sql_table"
matchedFieldValue: "pg_catalog"
matchedFieldLength: 18
matchedOffset: 4
matchedLength: 10
}

Maintaining privacy when verbose logging is turned on

When you use verbose logging, Google Cloud Armor logs snippets of the elements from the incoming requests that triggered a particular preconfigured WAF rule. These snippets might contain pieces of request headers, request parameters, or elements of the POST body. It is possible that a snippet contains sensitive data such as an IP address or other sensitive data from the incoming request, depending on what is in the request headers or body and what triggers the WAF rule.

By enabling verbose logging, be aware that there is a risk of accumulating potentially sensitive data in your logs in Logging. We recommend that you enable verbose logging only during rule creation and validation or for troubleshooting. During normal operations, we recommend that you leave verbose logging disabled.

Viewing logs

You can view the logs for a Google Cloud Armor security policy only in the Google Cloud Console.

Console

  1. In the Google Cloud Console, go to the Network Security page.

    Go to Network Security

  2. On the Security policies page, in the row for a security policy, click Menu for the policy whose logs you want to view.

  3. Select View logs.

Bot management

To give visibility into how Google Cloud Armor evaluates reCAPTCHA Enterprise tokens, jsonPayload has the following additional field:

  • securityPolicyRequestData: data pertaining to the request while being processed by a security policy, regardless of which rule is matched eventually.
    • recaptchaToken: data related to a reCAPTCHA Enterprise token (action-token or session-token).
      • score (float): a user legitimacy score embedded in a reCAPTCHA Enterprise token. Only present when a reCAPTCHA Enterprise token is attached with the request and the token is decoded correctly. Google Cloud Armor only attempts to decode the token if there is a reCAPTCHA Enterprise token-based rule in place. For more information, see Enforce reCAPTCHA Enterprise assessment.

Log examples

The following is an example of log details for a throttle rule that blocks a request:

jsonPayload: {
 enforcedSecurityPolicy: {
  priority: 100
  name: "sample-prod-policy"
  configuredAction: "THROTTLE"
  outcome: "THROTTLE"
 }
 @type: "type.googleapis.com/google.cloud.loadbalancing.type.LoadBalancerLogEntry"
 statusDetails: "throttled_by_security_policy"
}
httpRequest: {8}
resource: {2}
timestamp: "2021-03-17T19:16:48.185763Z"

The following is an example of log details for a rate-based ban rule that blocks a request:

jsonPayload: {
 @type: "type.googleapis.com/google.cloud.loadbalancing.type.LoadBalancerLogEntry"
 enforcedSecurityPolicy: {
  priority: 150
  name: "sample-prod-policy"
  outcome: "RATE_BASED_BAN"
  configuredAction: "RATE_BASED_BAN"
 }
 statusDetails: "rate_based_banned_by_security_policy"
}
httpRequest: {8}
resource: {2}
timestamp: "2021-03-17T19:27:17.393244Z"

What's next

Learn about Troubleshooting Google Cloud Armor issues.