Collect Google Cloud Load Balancing logs

Supported in:

This document describes how you can collect Google Cloud Load Balancing logs by enabling Google Cloud telemetry ingestion to Google Security Operations and how log fields map to Google Security Operations Unified Data Model (UDM) fields. This document also lists the supported Google Cloud Load Balancing version.

For more information, see Data ingestion to Google Security Operations.

A typical deployment consists of Google Cloud Load Balancing logs enabled for ingestion to Google Security Operations. Each customer deployment might differ from this representation and might be more complex.

The deployment contains the following components:

  • Google Cloud: The Google Cloud services and products from which you collect logs.

  • Google Cloud Load Balancing logs: The Google Cloud Load Balancing logs that are enabled for ingestion to Google Security Operations.

  • Google Security Operations: Google Security Operations retains and analyzes the logs from Google Cloud Load Balancing.

An ingestion label identifies the parser which normalizes raw log data to structured UDM format. The information in this document applies to the parser with the GCP_LOADBALANCING ingestion label.

Before you begin

  • Ensure that you are using Google Cloud Load Balancing version 1.

  • Ensure that all systems in the deployment architecture are configured in the UTC time zone.

Configure Google Cloud to ingest Google Cloud Load Balancing logs

To ingest Google Cloud Load Balancing logs to Google Security Operations, follow the steps on the Ingest Google Cloud logs to Google Security Operations page.

If you encounter issues when you ingest Google Cloud Load Balancing logs, contact Google Security Operations support.

Field mapping reference

This section explains how the Google Security Operations parser maps Google Cloud Load Balancing fields to Google Security Operations Unified Data Model (UDM) fields.

Field mapping reference: GCP_LOADBALANCING log fields to UDM fields

The following table lists the log fields of the GCP_LOADBALANCING log type and their corresponding UDM fields.

Log field UDM mapping Logic
metadata.event_type If the following values are not empty, then the metadata.event_type UDM field is set to NETWORK_CONNECTION.
  • httpRequest.remoteIp
  • jsonPayload.remoteIp
  • jsonPayload.connection.clientIp
  • jsonPayload.clientInstance.vmIp
  • httpRequest.serverIp
  • jsonPayload.connection.serverIp
  • jsonPayload.serverInstance.vmIp


Else, if the following values are not empty, then the metadata.event_type UDM field is set to STATUS_UNCATEGORIZED.
  • httpRequest.remoteIp
  • jsonPayload.remoteIp
  • jsonPayload.connection.clientIp
  • jsonPayload.clientInstance.vmIp


Else, the metadata.event_type UDM field is set to GENERIC_EVENT.
metadata.vendor_name The metadata.vendor_name UDM field is set to Google Cloud Platform.
principal.resource_ancestors.resource_subtype If the jsonPayload.clientVpc.projectId log field value is not empty, then the principal.resource_ancestors.resource_subtype UDM field is set to clientVpc_projectId.

If the jsonPayload.clientVpc.vpc log field value is not empty, then the principal.resource_ancestors.resource_subtype UDM field is set to clientVpc_vpc.

If the jsonPayload.clientVpc.subnetwork log field value is not empty, then the principal.resource_ancestors.resource_subtype UDM field is set to clientVpc_subnetwork.

If the jsonPayload.clientGkeDetails.cluster.cluster log field value is not empty, then the principal.resource_ancestors.resource_subtype UDM field is set to clientGkeDetails_cluster.

If the jsonPayload.clientGkeDetails.pod.pod log field value is not empty, then the principal.resource_ancestors.resource_subtype UDM field is set to clientGkeDetails_pod.

If the jsonPayload.clientGkeDetails.service.service log field value is not empty, then the principal.resource_ancestors.resource_subtype UDM field is set to clientGkeDetails_service.
principal.resource_ancestors.resource_type If the jsonPayload.clientVpc.projectId log field value is not empty, then the principal.resource_ancestors.resource_type UDM field is set to VPC_NETWORK.

If the jsonPayload.clientVpc.vpc log field value is not empty, then the principal.resource_ancestors.resource_type UDM field is set to VIRTUAL_MACHINE.

If the jsonPayload.clientVpc.subnetwork log field value is not empty, then the principal.resource_ancestors.resource_type UDM field is set to VIRTUAL_MACHINE.

If the jsonPayload.clientGkeDetails.cluster.cluster log field value is not empty, then the principal.resource_ancestors.resource_type UDM field is set to CLUSTER.

If the jsonPayload.clientGkeDetails.pod.pod log field value is not empty, then the principal.resource_ancestors.resource_type UDM field is set to VIRTUAL_MACHINE.

If the jsonPayload.clientGkeDetails.service.service log field value is not empty, then the principal.resource_ancestors.resource_type UDM field is set to BACKEND_SERVICE.
principal.resource.resource_subtype If the jsonPayload.clientInstance.vm log field value is not empty, then the principal.resource.resource_subtype UDM field is set to client_instance_vm.
principal.resource.resource_type If the jsonPayload.clientInstance.vm log field value is not empty, then the principal.resource.resource_type UDM field is set to VIRTUAL_MACHINE.
security_result.action If the jsonPayload.enforcedSecurityPolicy.configuredAction log field value is equal to DENY, then the security_result.action UDM field is set to BLOCK.

Else, if the jsonPayload.enforcedSecurityPolicy.configuredAction log field value is equal to ALLOW, then the security_result.action UDM field is set to ALLOW.
If the jsonPayload.previewSecurityPolicy.configuredAction log field value is equal to DENY, then the security_result.action UDM field is set to BLOCK.

Else, if the jsonPayload.previewSecurityPolicy.configuredAction log field value is equal to ALLOW, then the security_result.action UDM field is set to ALLOW.
If the jsonPayload.enforcedEdgeSecurityPolicy.configuredAction log field value is equal to DENY, then the security_result.action UDM field is set to BLOCK.

Else, if the jsonPayload.enforcedEdgeSecurityPolicy.configuredAction log field value is equal to ALLOW, then the security_result.action UDM field is set to ALLOW.
If the jsonPayload.previewEdgeSecurityPolicy.configuredAction log field value is equal to DENY, then the security_result.action UDM field is set to BLOCK.

Else, if the jsonPayload.previewEdgeSecurityPolicy.configuredAction log field value is equal to ALLOW, then the security_result.action UDM field is set to ALLOW.
security_result.severity If the severity log field value matches the regular expression DEFAULT or DEBUG or INFO or NOTICE, then the security_result.severity UDM field is set to LOW.

Else, if the severity log field value matches the regular expression WARNING or ERROR, then the security_result.severity UDM field is set to MEDIUM.

Else, if the severity log field value matches the regular expression CRITICAL or ALERT or EMERGENCY, then the security_result.severity UDM field is set to HIGH.
target.resource_ancestors.resource_type If the resource.labels.backend_target_name log field value is not empty, then the target.resource_ancestors.resource_type UDM field is set to BACKEND_SERVICE.

If the jsonPayload.serverInstance.vm log field value is not empty, then the target.resource_ancestors.resource_type UDM field is set to VIRTUAL_MACHINE.

If the jsonPayload.serverGkeDetails.cluster.cluster log field value is not empty, then the target.resource_ancestors.resource_type UDM field is set to CLUSTER.

If the jsonPayload.serverGkeDetails.pod.pod log field value is not empty, then the target.resource_ancestors.resource_type UDM field is set to VIRTUAL_MACHINE.

If the jsonPayload.serverGkeDetails.service.service log field value is not empty, then the target.resource_ancestors.resource_type UDM field is set to BACKEND_SERVICE.

If the resource.labels.network_name log field value is not empty, then the target.resource_ancestors.resource_type UDM field is set to VPC_NETWORK.
target.resource.attribute.cloud.environment The target.resource.attribute.cloud.environment UDM field is set to GOOGLE_CLOUD_PLATFORM.
target.resource.resource_type The target.resource.resource_type UDM field is set to DEVICE.
clientInstance.vmIp principal.ip
httpRequest.cacheFillBytes target.labels[http_request_cache_fill_bytes] (deprecated)
httpRequest.cacheFillBytes additional.fields[http_request_cache_fill_bytes]
httpRequest.cacheHit target.labels[http_request_cache_hit] (deprecated)
httpRequest.cacheHit additional.fields[http_request_cache_hit]
httpRequest.cacheLookup target.labels[http_request_cache_lookup] (deprecated)
httpRequest.cacheLookup additional.fields[http_request_cache_lookup]
httpRequest.cacheValidatedWithOriginServer target.labels[http_request_cache_validated_with_origin_server] (deprecated)
httpRequest.cacheValidatedWithOriginServer additional.fields[http_request_cache_validated_with_origin_server]
httpRequest.latency target.labels[http_request_latency] (deprecated)
httpRequest.latency additional.fields[http_request_latency]
httpRequest.protocol network.application_protocol If the httpRequest.requestUrl log field value matches the regular expression https or the httpRequest.protocol log field value matches the regular expression HTTPS, then the network.application_protocol UDM field is set to HTTPS.

Else, if the httpRequest.requestUrl log field value matches the regular expression http or the httpRequest.protocol log field value matches the regular expression HTTP, then the network.application_protocol UDM field is set to HTTP.
httpRequest.referer network.http.referral_url
httpRequest.remoteIp principal.ip If the httpRequest.remoteIp log field value is not empty, then
Grok: Extracted ip and port from the log field httpRequest.remoteIp and mapped it to the principal.ip and principal.port UDM field respectively.
httpRequest.requestMethod network.http.method
httpRequest.requestSize network.sent_bytes
httpRequest.requestUrl target.url
httpRequest.responseSize network.received_bytes
httpRequest.serverIp target.ip
httpRequest.status network.http.response_code
httpRequest.userAgent network.http.user_agent
insertId metadata.product_log_id
jsonPayload.@type about.labels[metadata_type] (deprecated)
jsonPayload.@type additional.fields[metadata_type]
jsonPayload.backendTargetProjectNumber about.labels[backend_target_project_number] (deprecated)
jsonPayload.backendTargetProjectNumber additional.fields[backend_target_project_number]
jsonPayload.bytesReceived network.received_bytes
jsonPayload.bytesSent network.sent_packets
jsonPayload.cacheDecision about.labels[cache_decision]
jsonPayload.cacheId about.labels[cache_id] (deprecated)
jsonPayload.cacheId additional.fields[cache_id]
jsonPayload.clientGkeDetails.cluster.cluster principal.resource_ancestors.name
jsonPayload.clientGkeDetails.cluster.clusterLocation principal.resource_ancestors.attribute.cloud.availability_zone
jsonPayload.clientGkeDetails.pod.pod principal.resource_ancestors.name
jsonPayload.clientGkeDetails.pod.podNamespace principal.resource_ancestors.attribute.labels[pod_namespace]
jsonPayload.clientGkeDetails.service.service principal.resource_ancestors.name
jsonPayload.clientGkeDetails.service.serviceNamespace principal.resource_ancestors.attribute.labels[service_namespace]
jsonPayload.clientInstance.projectId principal.resource_ancestors.product_object_id
jsonPayload.clientInstance.region principal.resource.attribute.labels[client_instance_region]
jsonPayload.clientInstance.vm principal.resource.name
jsonPayload.clientInstance.zone principal.resource.attribute.cloud.availability_zone
jsonPayload.clientLocation.asn network.asn
jsonPayload.clientLocation.city principal.location.city
jsonPayload.clientLocation.continent principal.labels[client_loacation_continent] (deprecated)
jsonPayload.clientLocation.continent additional.fields[client_loacation_continent]
jsonPayload.clientLocation.regionCode principal.location.country_or_region
jsonPayload.clientLocation.subRegion principal.location.state
jsonPayload.clientVpc.projectId principal.resource_ancestors.name
jsonPayload.clientVpc.subnetwork principal.resource_ancestors.name
jsonPayload.clientVpc.vpc principal.resource_ancestors.name
jsonPayload.connection.clientIp principal.ip
jsonPayload.connection.clientPort principal.port
jsonPayload.connection.protocol network.ip_protocol If the jsonPayload.connection.protocol log field value is equal to 0, then the network.ip_protocol UDM field is set to UNKNOWN_IP_PROTOCOL.

Else, if the jsonPayload.connection.protocol log field value is equal to 1, then the network.ip_protocol UDM field is set to ICMP.

Else, if the jsonPayload.connection.protocol log field value is equal to 2, then the network.ip_protocol UDM field is set to IGMP.

Else, if the jsonPayload.connection.protocol log field value is equal to 6, then the network.ip_protocol UDM field is set to TCP.

Else, if the jsonPayload.connection.protocol log field value is equal to 17, then the network.ip_protocol UDM field is set to UDP.

Else, if the jsonPayload.connection.protocol log field value is equal to 41, then the network.ip_protocol UDM field is set to IP6IN4.

Else, if the jsonPayload.connection.protocol log field value is equal to 47, then the network.ip_protocol UDM field is set to GRE.

Else, if the jsonPayload.connection.protocol log field value is equal to 50, then the network.ip_protocol UDM field is set to ESP.

Else, if the jsonPayload.connection.protocol log field value is equal to 58, then the network.ip_protocol UDM field is set to ICMP6.

Else, if the jsonPayload.connection.protocol log field value is equal to 88, then the network.ip_protocol UDM field is set to EIGRP.

Else, if the jsonPayload.connection.protocol log field value is equal to 97, then the network.ip_protocol UDM field is set to ETHERIP.

Else, if the jsonPayload.connection.protocol log field value is equal to 103, then the network.ip_protocol UDM field is set to PIM.

Else, if the jsonPayload.connection.protocol log field value is equal to 112, then the network.ip_protocol UDM field is set to VRRP.

Else, if the jsonPayload.connection.protocol log field value is equal to 132, then the network.ip_protocol UDM field is set to SCTP.
jsonPayload.connection.serverIp target.ip
jsonPayload.connection.serverPort target.port
jsonPayload.endTime about.labels[end_time] (deprecated)
jsonPayload.endTime additional.fields[end_time]
jsonPayload.enforcedEdgeSecurityPolicy.configuredAction security_result.action_details
jsonPayload.enforcedEdgeSecurityPolicy.name security_result.rule_name
jsonPayload.enforcedEdgeSecurityPolicy.outcome security_result.outcomes[enforcededgesecuritypolicy_outcome]
jsonPayload.enforcedEdgeSecurityPolicy.priority security_result.priority_details
jsonPayload.enforcedSecurityPolicy.adaptiveProtection.autoDeployAlertId security_result.rule_labels[adaptiveprotection_autodeployalertid]
jsonPayload.enforcedSecurityPolicy.configuredAction security_result.action_details
jsonPayload.enforcedSecurityPolicy.matchedFieldName security_result.rule_labels[matched_field_name]
jsonPayload.enforcedSecurityPolicy.matchedFieldType security_result.rule_labels[matched_field_type]
jsonPayload.enforcedSecurityPolicy.matchedFieldValue security_result.rule_labels[matched_field_value]
jsonPayload.enforcedSecurityPolicy.matchedLength security_result.rule_labels[matched_length]
jsonPayload.enforcedSecurityPolicy.name security_result.rule_name
jsonPayload.enforcedSecurityPolicy.outcome security_result.outcomes[jsonpayload_enforcedsecuritypolicy_outcome]
jsonPayload.enforcedSecurityPolicy.preconfiguredExprIds security_result.rule_labels[preconfigured_expr_ids]
jsonPayload.enforcedSecurityPolicy.priority security_result.priority_details
jsonPayload.enforcedSecurityPolicy.rateLimitAction.key security_result.rule_labels[enforcedsecuritypolicy_ratelimitaction_key]
jsonPayload.enforcedSecurityPolicy.rateLimitAction.outcome security_result.rule_labels[enforcedsecuritypolicy_ratelimitaction_outcome]
jsonPayload.enforcedSecurityPolicy.threatIntelligence.categories security_result.rule_labels[threat_intelligence_category]
jsonPayload.networkTier.networkTier principal.labels[network_tier] (deprecated)
jsonPayload.networkTier.networkTier additional.fields[network_tier]
jsonPayload.packetsReceived network.received_packets
jsonPayload.packetsSent network.sent_packets
jsonPayload.previewEdgeSecurityPolicy.configuredAction security_result.action_details
jsonPayload.previewEdgeSecurityPolicy.name security_result.rule_name
jsonPayload.previewEdgeSecurityPolicy.outcome security_result.outcomes[previewedgesecuritypolicy_outcome]
jsonPayload.previewEdgeSecurityPolicy.priority security_result.priority_details
jsonPayload.previewSecurityPolicy.configuredAction security_result.action_details
jsonPayload.previewSecurityPolicy.name security_result.rule_name
jsonPayload.previewSecurityPolicy.outcome security_result.outcomes[previewsecuritypolicy_outcome]
jsonPayload.previewSecurityPolicy.preconfiguredExprIds security_result.rule_labels[previewsecuritypolicy_preconfigured_expr_ids]
jsonPayload.previewSecurityPolicy.priority security_result.priority_details
jsonPayload.previewSecurityPolicy.rateLimitAction.key security_result.rule_labels[previewsecuritypolicy_ratelimitaction_key]
jsonPayload.previewSecurityPolicy.rateLimitAction.outcome security_result.rule_labels[previewsecuritypolicy_ratelimitaction_outcome]
jsonPayload.proxyStatus security_result.summary
jsonPayload.remoteIp principal.ip If the jsonPayload.remoteIp log field value is not empty, then
Grok: Extracted ip and port from the log field jsonPayload.remoteIp and mapped it to the principal.ip and principal.port UDM field respectively.
jsonPayload.rtt network.session_duration.seconds Grok: Extracted sec from the log field jsonPayload.rtt and mapped it to the network.session_duration.seconds UDM field.
jsonPayload.rtt network.session_duration.nanos Grok: Extracted nano from the log field jsonPayload.rtt and mapped it to the network.session_duration.nanos UDM field.
jsonPayload.securityPolicyRequestData.recaptchaActionToken.score security_result.risk_score If the jsonPayload.securityPolicyRequestData.recaptchaActionToken.score log field value is not empty, then the jsonPayload.securityPolicyRequestData.recaptchaActionToken.score log field is mapped to the security_result.risk_score UDM field.
jsonPayload.securityPolicyRequestData.recaptchaSessionToken.score security_result.risk_score If the jsonPayload.securityPolicyRequestData.recaptchaSessionToken.score log field value is not empty, then the jsonPayload.securityPolicyRequestData.recaptchaSessionToken.score log field is mapped to the security_result.risk_score UDM field.
jsonPayload.securityPolicyRequestData.remoteIpInfo.regionCode principal.location.name
jsonPayload.securityPolicyRequestData.tlsJa3Fingerprint network.tls.client.ja3
jsonPayload.serverGkeDetails.cluster.cluster target.resource_ancestors.name
jsonPayload.serverGkeDetails.cluster.clusterLocation target.resource_ancestors.attribute.cloud.availability_zone If the jsonPayload.serverGkeDetails.cluster.cluster log field value is not empty, then the jsonPayload.serverGkeDetails.cluster.clusterLocation log field is mapped to the target.resource_ancestors.attribute.cloud.availability_zone UDM field.
jsonPayload.serverGkeDetails.pod.pod target.resource_ancestors.name
jsonPayload.serverGkeDetails.pod.podNamespace target.resource_ancestors.attribute.labels[pod_namespace]
jsonPayload.serverGkeDetails.service.service target.resource_ancestors.name
jsonPayload.serverGkeDetails.service.serviceNamespace target.resource_ancestors.attribute.labels[service_namespace]
jsonPayload.serverInstance.projectId target.resource_ancestors.product_object_id If the jsonPayload.serverInstance.vm log field value is not empty, then the jsonPayload.serverInstance.projectId log field is mapped to the target.resource_ancestors.product_object_id UDM field.
jsonPayload.serverInstance.region target.resource_ancestors.attribute.labels[client_instance_region]
jsonPayload.serverInstance.vm target.resource_ancestors.name
jsonPayload.serverInstance.zone target.resource_ancestors.attribute.cloud.availability_zone If the jsonPayload.serverInstance.vm log field value is not empty, then the jsonPayload.serverInstance.zone log field is mapped to the target.resource_ancestors.attribute.cloud.availability_zone UDM field.
jsonPayload.startTime about.labels[start_time] (deprecated)
jsonPayload.startTime additional.fields[start_time]
jsonPayload.statusDetails security_result.summary
jsonPayload.tls.cipher network.tls.cipher
jsonPayload.tls.protocol network.tls.next_protocol
logName metadata.product_event_type
receiveTimestamp metadata.collected_timestamp
resource.labels.backend_failover_configuration target.resource_ancestors.attribute.labels[backend_failover_configuration]
resource.labels.backend_group_name target.group.group_display_name
resource.labels.backend_group_scope target.group.attribute.labels[backend_group_scope]
resource.labels.backend_group_type target.group.attribute.labels[backend_group_type]
resource.labels.backend_name target.group.group_display_name
resource.labels.backend_network_name target.resource_ancestors.attribute.labels[backend_network_name]
resource.labels.backend_scope target.resource_ancestors.attribute.cloud.availability_zone If the resource.labels.backend_target_name log field value is not empty, then the resource.labels.backend_scope log field is mapped to the target.resource_ancestors.attribute.cloud.availability_zone UDM field.
resource.labels.backend_scope_type target.resource_ancestors.attribute.labels[backend_scope_type]
resource.labels.backend_service_name target.application
resource.labels.backend_subnetwork_name target.resource_ancestors.attribute.labels[backend_subnetwork_name]
resource.labels.backend_target_name target.resource_ancestors.name
resource.labels.backend_target_type target.resource_ancestors.resource_subtype If the resource.labels.backend_target_name log field value is not empty, then the resource.labels.backend_target_type log field is mapped to the target.resource_ancestors.resource_subtype UDM field.

If the jsonPayload.serverInstance.vm log field value is not empty, then the target.resource_ancestors.resource_subtype UDM field is set to serverInstance_vm.

If the jsonPayload.serverGkeDetails.cluster.cluster log field value is not empty, then the target.resource_ancestors.resource_subtype UDM field is set to serverGkeDetails_cluster.

If the jsonPayload.serverGkeDetails.pod.pod log field value is not empty, then the target.resource_ancestors.resource_subtype UDM field is set to serverGkeDetails_pod.

If the jsonPayload.serverGkeDetails.service.service log field value is not empty, then the target.resource_ancestors.resource_subtype UDM field is set to serverGkeDetails_service.

If the resource.labels.network_name log field value is not empty, then the target.resource_ancestors.resource_subtype UDM field is set to network_name.
resource.labels.backend_type target.group.attribute.labels[backend_type]
resource.labels.backend_zone target.resource_ancestors.attribute.cloud.availability_zone If the resource.labels.backend_zone log field value is not empty, then the resource.labels.backend_zone log field is mapped to the target.resource_ancestors.attribute.cloud.availability_zone UDM field.
resource.labels.endpoint_zone target.resource.attribute.cloud.availability_zone
resource.labels.forwarding_rule_name security_result.rule_labels[forwarding_rule_name]
resource.labels.forwarding_rule_network_tier target.labels[forwarding_rule_network_tier] (deprecated)`
resource.labels.forwarding_rule_network_tier additional.fields[forwarding_rule_network_tier]
resource.labels.load_balancer_name target.resource.name
resource.labels.loadbalancing_scheme_name target.resource.attribute.labels[loadbalancing_scheme_name]
resource.labels.matched_url_path_rule target.resource.attribute.labels[matched_url_path_rule]
resource.labels.network_name target.resource_ancestors.name
resource.labels.primary_target_pool target.labels[primary_target_pool] (deprecated)
resource.labels.primary_target_pool additional.fields[primary_target_pool]
resource.labels.project target.resource_ancestors.product_object_id
resource.labels.project_id target.resource_ancestors.product_object_id
resource.labels.region target.resource.attribute.cloud.availability_zone
resource.labels.target_pool target.labels[target_pool] (deprecated)
resource.labels.target_pool additional.fields[target_pool]
resource.labels.target_proxy_name target.labels[target_proxy_name] (deprecated)
resource.labels.target_proxy_name additional.fields[target_proxy_name]
resource.labels.url_map_name target.labels[url_map_name] (deprecated)
resource.labels.url_map_name additional.fields[url_map_name]
resource.labels.zone target.resource.attribute.cloud.availability_zone
resource.type target.resource.resource_subtype
serverInstance.vmIp target.ip
severity security_result.severity_details
spanId about.labels[span_id] (deprecated)
spanId additional.fields[span_id]
timestamp metadata.event_timestamp
trace about.labels[trace] (deprecated)
trace additional.fields[trace]
traceSampled about.labels[trace_sampled] (deprecated)
traceSampled additional.fields[trace_sampled]

What's next

Need more help? Get answers from Community members and Google SecOps professionals.