Google Cloud
Overview
This parser code extracts data from various Google Cloud logs in JSON format. It identifies the specific Google Cloud service based on the _resourceType field and then uses specialized include files to extract relevant fields and map them to a unified data model (UDM) for consistent security analysis.
UDM Mapping Table
| Log Field | UDM Mapping | Logic |
|---|---|---|
| authenticationInfo.principalEmail | principal.user.email_addresses | Directly mapped from the log field. |
| authenticationInfo.principalSubject | principal.user.email_addresses | Directly mapped from the log field. |
| authenticationInfo.serviceAccountKeyName | principal.user.attribute .key_id |
Directly mapped from the log field. |
| authorizationInfo[].granted | security_result.action | If any authorizationInfo[].granted is true, then action is ALLOW. |
| authorizationInfo[].permission | principal.user.attribute .permissions.name |
Directly mapped from the log field. |
| authorizationInfo[].resource | security_result.detection_fields.resource | Directly mapped from the log field. |
| authorizationInfo[].resourceAttributes.name | principal.user.attribute .cloud.project.name |
Directly mapped from the log field. |
| authorizationInfo[].resourceAttributes.service | principal.user.attribute .cloud.project.resource_subtype |
Directly mapped from the log field. |
| authorizationInfo[].resourceAttributes.type | principal.user.attribute .cloud.project.resource_subtype |
Directly mapped from the log field. |
| insertId | metadata.product_log_id | Directly mapped from the log field. |
| jsonPayload.actor.user | principal.user.email_addresses | Directly mapped from the log field. |
| jsonPayload.authAnswer | network.dns.authoritative | Directly mapped from the log field. |
| jsonPayload.bytes_sent | network.sent_bytes | Directly mapped from the log field. |
| jsonPayload.connection.dest_ip | target.ip | Directly mapped from the log field. |
| jsonPayload.connection.dest_port | target.port | Directly mapped from the log field. |
| jsonPayload.connection.nat_ip | principal.nat_ip | Directly mapped from the log field. |
| jsonPayload.connection.nat_port | principal.nat_port | Directly mapped from the log field. |
| jsonPayload.connection.protocol | network.ip_protocol | Mapped from the log field. 6 represents TCP, 17 represents UDP, 50 represents ESP. |
| jsonPayload.connection.src_ip | principal.ip | Directly mapped from the log field. |
| jsonPayload.connection.src_port | principal.port | Directly mapped from the log field. |
| jsonPayload.dest_instance.project_id | target.cloud.project.name | Directly mapped from the log field. |
| jsonPayload.dest_instance.region | target.cloud.availability_zone | Directly mapped from the log field. |
| jsonPayload.dest_instance.vm_name | target.hostname | Directly mapped from the log field. |
| jsonPayload.dest_instance.zone | target.cloud.availability_zone | Directly mapped from the log field. |
| jsonPayload.dest_location.Location | target.location.name | Directly mapped from the log field. |
| jsonPayload.dest_location.asn | target.location.asn | Directly mapped from the log field. |
| jsonPayload.dest_location.continent | target.location.continent | Directly mapped from the log field. |
| jsonPayload.dest_vpc.project_id | target.cloud.project.name | Directly mapped from the log field. |
| jsonPayload.dest_vpc.subnetwork_name | target.cloud.vpc.name | Directly mapped from the log field. |
| jsonPayload.dest_vpc.vpc_name | target.cloud.vpc.name | Directly mapped from the log field. |
| jsonPayload.event_subtype | metadata.product_event_type | Directly mapped from the log field. |
| jsonPayload.event_type | metadata.event_type | Mapped from the log field. GCE_OPERATION_DONE is mapped to RESOURCE_CREATION or RESOURCE_DELETION based on the event_subtype. |
| jsonPayload.operation.id | target.cloud.resource.product_object_id | Directly mapped from the log field. |
| jsonPayload.protocol | network.ip_protocol | Mapped from the log field. 6 represents TCP, 17 represents UDP, 50 represents ESP. |
| jsonPayload.queryName | network.dns.questions.name | Directly mapped from the log field. |
| jsonPayload.queryType | network.dns.questions.type | Mapped from the log field. A is mapped to 1, AAAA is mapped to 28. |
| jsonPayload.rdata | network.dns.answers.data | Directly mapped from the log field. |
| jsonPayload.responseCode | additional.fields.response_code .string_value |
Directly mapped from the log field. |
| jsonPayload.resource.id | target.cloud.resource.product_object_id | Directly mapped from the log field. |
| jsonPayload.resource.name | target.cloud.resource.name | Directly mapped from the log field. |
| jsonPayload.resource.type | target.cloud.resource.resource_type | Directly mapped from the log field. |
| jsonPayload.sourceIP | principal.ip | Directly mapped from the log field. |
| jsonPayload.sourceNetwork | principal.namespace | Directly mapped from the log field. |
| jsonPayload.src_instance.project_id | principal.cloud.project.name | Directly mapped from the log field. |
| jsonPayload.src_instance.region | principal.cloud.availability_zone | Directly mapped from the log field. |
| jsonPayload.src_instance.vm_name | principal.hostname | Directly mapped from the log field. |
| jsonPayload.src_instance.zone | principal.cloud.availability_zone | Directly mapped from the log field. |
| jsonPayload.src_location.asn | principal.location.asn | Directly mapped from the log field. |
| jsonPayload.src_location.city | principal.location.city | Directly mapped from the log field. |
| jsonPayload.src_location.continent | principal.location.continent | Directly mapped from the log field. |
| jsonPayload.src_location.country | principal.location.country | Directly mapped from the log field. |
| jsonPayload.src_location.region | principal.location.region | Directly mapped from the log field. |
| jsonPayload.src_vpc.project_id | principal.cloud.project.name | Directly mapped from the log field. |
| jsonPayload.src_vpc.subnetwork_name | principal.cloud.vpc.name | Directly mapped from the log field. |
| jsonPayload.src_vpc.vpc_name | principal.cloud.vpc.name | Directly mapped from the log field. |
| jsonPayload.vmInstanceName | principal.cloud.resource .name |
Directly mapped from the log field. |
| labels ."compute.googleapis.com/resource_id" |
target.cloud.resource.product_object_id | Directly mapped from the log field. |
| labels ."compute.googleapis.com/resource_name" |
target.cloud.resource.name | Directly mapped from the log field. |
| labels ."compute.googleapis.com/resource_type" |
target.cloud.resource.resource_type | Directly mapped from the log field. |
| labels.cluster_name | target.cloud.resource.name | Directly mapped from the log field. |
| labels.cluster_uuid | target.cloud.resource.product_object_id | Directly mapped from the log field. |
| labels.database_id | target.cloud.resource .attribute.labels.database_id |
Directly mapped from the log field. |
| labels.dataset_id | target.cloud.resource.name | Directly mapped from the log field. |
| labels.email_id | target.user.email_addresses | Directly mapped from the log field. |
| labels.firewall_rule_id | target.cloud.resource.product_object_id | Directly mapped from the log field. |
| labels.forwarding_rule_id | target.cloud.resource.product_object_id | Directly mapped from the log field. |
| labels.instance_group_name | target.cloud.resource.name | Directly mapped from the log field. |
| labels.instance_id | target.cloud.resource.product_object_id | Directly mapped from the log field. |
| labels.location | target.cloud.availability_zone | Directly mapped from the log field. |
| labels.method | security_result.category_details | Directly mapped from the log field. |
| labels.project_id | target.cloud.project.name | Directly mapped from the log field. |
| labels.region | target.cloud.availability_zone | Directly mapped from the log field. |
| labels.service | target.application | Directly mapped from the log field. |
| labels.subnetwork_id | target.cloud.vpc.id | Directly mapped from the log field. |
| labels.subnetwork_name | target.cloud.vpc.name | Directly mapped from the log field. |
| labels.unique_id | target.cloud.resource.product_object_id | Directly mapped from the log field. |
| labels.zone | target.cloud.availability_zone | Directly mapped from the log field. |
| logName | security_result.category_details | Directly mapped from the log field. |
| metadata.event[].eventName | target.cloud.resource.attribute.labels.'event name' | Directly mapped from the log field. |
| metadata.event[].eventType | target.cloud.resource.attribute.labels.'event type' | Directly mapped from the log field. |
| methodName | metadata.product_event_type | Directly mapped from the log field. |
| operation.id | security_result.detection_fields.resource_name | Directly mapped from the log field. |
| operation.producer | target.application | Directly mapped from the log field. |
| protoPayload.authenticationInfo .principalEmail |
principal.user.email_addresses | Directly mapped from the log field. |
| protoPayload.authenticationInfo .principalSubject |
principal.user.email_addresses | Directly mapped from the log field. |
| protoPayload.authenticationInfo .serviceAccountKeyName |
principal.user.attribute.key_id | Directly mapped from the log field. |
| protoPayload.authorizationInfo[] .granted |
security_result.action | If any authorizationInfo[].granted is true, then action is ALLOW. |
| protoPayload.authorizationInfo[] .permission |
principal.user.attribute .permissions.name |
Directly mapped from the log field. |
| protoPayload.authorizationInfo[] .resource |
security_result.detection_fields.resource | Directly mapped from the log field. |
| protoPayload.authorizationInfo[] .resourceAttributes.name |
principal.user.attribute .cloud.project.name |
Directly mapped from the log field. |
| protoPayload.authorizationInfo[] .resourceAttributes.service |
principal.user.attribute .cloud.project.resource_subtype |
Directly mapped from the log field. |
| protoPayload.authorizationInfo[] .resourceAttributes.type |
principal.user.attribute .cloud.project.resource_subtype |
Directly mapped from the log field. |
| protoPayload.metadata .datasetCreation.dataset.acl .policy.bindings[].members |
target.user.email_addresses | Directly mapped from the log field. |
| protoPayload.metadata .datasetCreation.dataset .datasetName |
security_result.detection_fields.resource_name | Directly mapped from the log field. |
| protoPayload.metadata .event[].eventName |
target.cloud.resource .attribute.labels.'event name' |
Directly mapped from the log field. |
| protoPayload.metadata .event[].eventType |
target.cloud.resource .attribute.labels.'event type' |
Directly mapped from the log field. |
| protoPayload.metadata .jobInsertion.job.jobConfig .queryConfig.destinationTable |
target.cloud.resource.name | Directly mapped from the log field. |
| protoPayload.metadata .jobInsertion.job.jobConfig .queryConfig.query |
target.process.command_line | Directly mapped from the log field. |
| protoPayload.metadata .jobInsertion.job.jobName |
security_result.detection_fields .resource_name |
Directly mapped from the log field. |
| protoPayload.metadata .jobInsertion.job.jobStatus .jobState |
security_result.description | Directly mapped from the log field. |
| protoPayload.metadata .tableDeletion.reason |
security_result.description | Directly mapped from the log field. |
| protoPayload.methodName | metadata.product_event_type | Directly mapped from the log field. |
| protoPayload.request."@type" | principal.user.attribute .cloud.project.resource_subtype |
Directly mapped from the log field. |
| protoPayload.request.accessLevel .name |
target.cloud.resource.name | Directly mapped from the log field. |
| protoPayload.request.accessLevel .title |
target.cloud.resource.name | Directly mapped from the log field. |
| protoPayload.request.alloweds[] .IPProtocol |
security_result.detection_fields .'allowed:ipprotocol' |
Directly mapped from the log field. |
| protoPayload.request.alloweds[] .ports |
security_result.detection_fields .'allowed:ports:tcp'.value |
Directly mapped from the log field. |
| protoPayload.request.consumerProjectId | target.cloud.project.name | Directly mapped from the log field. |
| protoPayload.request.destinationRanges | target.cloud.resource .attribute.labels.destination_ranges |
Directly mapped from the log field. |
| protoPayload.request.direction | target.cloud.resource.attribute.labels .direction |
Directly mapped from the log field. |
| protoPayload.request.disabled | target.cloud.resource .attribute.labels.'Request Disabled' |
Directly mapped from the log field. |
| protoPayload.request.entity[] .key.path.element[].name |
target.cloud.resource .attribute.labels.User |
Directly mapped from the log field. |
| protoPayload.request.entity[] .key.path.element[].type |
target.cloud.resource .attribute.labels.Fellow |
Directly mapped from the log field. |
| protoPayload.request .instances[].instance |
principal.user.attribute .cloud.project.name |
Directly mapped from the log field. |
| protoPayload.request.job .jobConfig.query.query |
target.process.command_line | Directly mapped from the log field. |
| protoPayload.request.job .jobStatus.state |
security_result.description | Directly mapped from the log field. |
| protoPayload.request.logConfig .enable |
target.cloud.resource .attribute.labels.'logConfig:enable' |
Directly mapped from the log field. |
| protoPayload.request.name | principal.user.attribute .cloud.project.name |
Directly mapped from the log field. |
| protoPayload.request.name | target.cloud.resource.attribute .labels.request_name |
Directly mapped from the log field. |
| protoPayload.request.network | target.cloud.resource.attribute .labels.'Request Network' |
Directly mapped from the log field. |
| protoPayload.request.priority | target.cloud.resource.attribute .labels.'Request Priority' |
Directly mapped from the log field. |
| protoPayload.request.requestId | principal.user.attribute .cloud.project.name |
Directly mapped from the log field. |
| protoPayload.request .serviceAccounts[].email |
target.user.email_addresses | Directly mapped from the log field. |
| protoPayload.request .serviceAccounts[].scopes |
security_result.detection_fields .service_account_scope |
Directly mapped from the log field. |
| protoPayload.requestMetadata .callerIp |
principal.ip | Directly mapped from the log field. |
| protoPayload.requestMetadata .callerSuppliedUserAgent |
network.http.user_agent | Directly mapped from the log field. |
| protoPayload.requestMetadata .requestAttributes.reason |
principal.user.attribute .labels.request_attributes_reason |
Directly mapped from the log field. |
| protoPayload.requestMetadata .requestAttributes.time |
principal.user.attribute .labels.request_attributes_time |
Directly mapped from the log field. |
| protoPayload.resourceName | security_result.detection_fields .resource_name |
Directly mapped from the log field. |
| protoPayload.response.email | target.user.email_addresses | Directly mapped from the log field. |
| protoPayload.response.error .code |
security_result.detection_fields .'Status Code' |
Directly mapped from the log field. |
| protoPayload.response.error .message |
security_result.summary | Directly mapped from the log field. |
| protoPayload.response.id | target.cloud.resource .attribute.labels.response_name |
Directly mapped from the log field. |
| protoPayload.response.name | target.cloud.resource .attribute.labels.response_name |
Directly mapped from the log field. |
| protoPayload.response.operationType | target.cloud.resource.attribute .labels.response_operation_type |
Directly mapped from the log field. |
| protoPayload.response.status | target.cloud.resource .attribute.labels.response_status |
Directly mapped from the log field. |
| protoPayload.response.zone | target.cloud.resource .attribute.labels.response_zone |
Directly mapped from the log field. |
| protoPayload.serviceData .jobCompletedEvent.job.jobConfiguration .query.destinationTable.datasetId |
target.cloud.resource.parent | Directly mapped from the log field. |
| protoPayload.serviceData .jobCompletedEvent.job.jobConfiguration .query.destinationTable.projectId |
target.cloud.project.name | Directly mapped from the log field. |
| protoPayload.serviceData .jobCompletedEvent.job.jobConfiguration .query.destinationTable.tableId |
target.cloud.resource.name | Directly mapped from the log field. |
| protoPayload.serviceData .jobCompletedEvent.job.jobConfiguration .query.query |
target.process.command_line | Directly mapped from the log field. |
| protoPayload.serviceData .jobCompletedEvent.job.jobStatistics .referencedTables[].datasetId |
target.cloud.resource.parent | Directly mapped from the log field. |
| protoPayload.serviceData .jobCompletedEvent.job.jobStatistics .referencedTables[].projectId |
target.cloud.project.name | Directly mapped from the log field. |
| protoPayload.serviceData .jobCompletedEvent.job.jobStatistics .referencedTables[].tableId |
target.cloud.resource.name | Directly mapped from the log field. |
| protoPayload.service |
Changes
2023-07-28
- Enhancement -
- Mapped "protoPayload.request.entity.0.key.path.element.type" to "target.resource.attribute.labels.key" and Mapped "protoPayload.request.entity.0.key.path.element.name" to "target.resource.attribute.labels.value".
2022-09-21
- Enhancement -
- Mapped the field 'severity' to 'security_result.severity' and 'security_result.severity_details'.