Collect AWS CloudTrail logs

Supported in:

Google SecOps SIEM

This document details the steps for configuring the ingestion of AWS CloudTrail logs and context data into Google Security Operations. These steps also apply to ingesting logs from other AWS services, such as AWS GuardDuty, AWS VPC Flow, AWS CloudWatch, and AWS Security Hub.

To ingest event logs, the configuration directs the CloudTrail logs into an Amazon Simple Storage Service (Amazon S3) bucket. You have the option of choosing either Amazon Simple Queue Service (Amazon SQS) or Amazon S3 as the feed source type.

The first part of this document provides concise steps for using Amazon S3 as the feed source type or, preferably using Amazon S3 with Amazon SQS as the feed source type. The second part provides more detailed steps with screenshots for using Amazon S3 as the feed source type. Using Amazon SQS is not covered in the second part. The third part provides information about how to ingest AWS context data about hosts, services, VPC networks, and users.

Basic steps to ingest logs from S3 with or without SQS

This section describes the basic steps for ingesting AWS CloudTrail logs into your Google Security Operations instance. The steps describe how to do this using Amazon S3 with Amazon SQS as the feed source type or, optionally, using Amazon S3 as the feed source type.

Configure AWS CloudTrail and S3

In this procedure, you configure AWS CloudTrail logs to be written to an S3 bucket.

In the AWS console, search for CloudTrail.
Click Create trail.
Provide a Trail name.
Select Create new S3 bucket. You may also choose to use an existing S3 bucket.
Provide a name for AWS KMS alias, or choose an existing AWS KMS Key.
You can leave the other settings as default, and click Next.
Choose Event type, add Data events as required, and click Next.
Review the settings in Review and create and click Create trail.
In the AWS console, search for Amazon S3 Buckets.
Click the newly created log bucket, and select the folder AWSLogs. Then click Copy S3 URI and save it for use in the following steps.

Create an SQS queue

It is recommended to use an SQS queue. If you use an SQS queue, it must be a Standard queue, not a FIFO queue.

For details about creating SQS queues, see Getting started with Amazon SQS.

Set up notifications to your SQS queue

If you use an SQS queue, set up notifications on your S3 bucket to write to your SQS queue. Be sure to attach an access policy.

Configure AWS IAM user

Configure an AWS IAM user which Google Security Operations will use to access both the SQS queue (if used) and the S3 bucket.

In the AWS console, search for IAM.
Click Users, and then in the following screen, click Add Users.
Provide a name for the user, for example, chronicle-feed-user, Select AWS credential type as Access key - Programmatic access and click Next: Permissions.
In the next step, select Attach existing policies directly and select AmazonS3ReadOnlyAccess or AmazonS3FullAccess, as required. AmazonS3FullAccess would be used if Google Security Operations should clear the S3 buckets after reading logs, to optimize AWS S3 storage costs.
As a recommended alternative to the previous step, you can further restrict access to only the specified S3 bucket by creating a custom policy. Click Create policy and follow the AWS documentation to create a custom policy.
When you apply a policy, make sure that you have included sqs:DeleteMessage. Google Security Operations is not able to delete messages if the sqs:DeleteMessage permission is not attached to the SQS queue. All the messages are accumulated on the AWS side, which causes a delay as Google Security Operations repeatedly attempts to transfer the same files.
Click Next:Tags.
Add any tags if required, and click Next:Review.
Review the configuration and click Create user.
Copy the Access key ID and Secret access key of the created user, for use in the next step.

Create the feed

After completing the preceding procedures, create a feed to ingest AWS logs from your Amazon S3 bucket into your Google Security Operations instance. If you are not using an SQS queue, in the following procedure select Amazon S3 for the feed source type instead of Amazon SQS.

To create a feed:

In the navigation bar, select Settings > SIEM Settings, and then Feeds.
On the Feeds page, click Add New.
In the Add feed dialog, use the Source type dialog to select either Amazon SQS or Amazon S3.
In the Log Type menu, select AWS CloudTrail (or another AWS service).
Click Next.
Enter the input parameters for your feed in the fields.
If the feed source type is Amazon S3, do the following:
1. Select region and provide S3 URI of the Amazon S3 bucket you copied earlier. You can also append the S3 URI using the variable.
```
 {{datetime("yyyy/MM/dd")}}
 
```
  In the following example, Google Security Operations scans logs each time only for a particular day.
```
 s3://aws-cloudtrail-logs-XXX-1234567/AWSLogs/1234567890/CloudTrail/us-east-1/{{datetime("yyyy/MM/dd")}}/
 
```
2. For URI IS A, select Directories including subdirectories. Select an appropriate option under Source Deletion Option. Ensure it matches the permissions of the IAM User account you created earlier.
3. Provide the Access Key ID and Secret Access Key of the IAM user account you created earlier.
Click Next and Finish.

Detailed steps to ingest logs from S3

Configure AWS CloudTrail (or other service)

Complete the following steps to configure AWS CloudTrail logs and direct these logs to be written to the AWS S3 bucket created in the previous procedure:

In the AWS console, search for CloudTrail.
Click Create trail.
Provide a Trail name.
Select Create new S3 bucket. You may also choose to use an existing S3 bucket.
Provide a name for AWS KMS alias, or choose an existing AWS KMS Key.
You can leave the other settings as default, and click Next.
Choose Event type, add Data events as required, and click Next.
Review the settings in Review and create and click Create trail.
In the AWS console, search for Amazon S3 Buckets.
Click the newly created log bucket, and select the folder AWSLogs. Then click Copy S3 URI and save it for use in the following steps.

Configure AWS IAM User

In this step, we will configure an AWS IAM user which Google Security Operations will use to get log feeds from AWS.

In the AWS console, search for IAM.
Click Users, and then in the following screen, click Add Users.
Provide a name for the user, for example, chronicle-feed-user, Select AWS credential type as Access key - Programmatic access and click Next: Permissions.
In the next step, select Attach existing policies directly and select AmazonS3ReadOnlyAccess or AmazonS3FullAccess, as required. AmazonS3FullAccess would be used if Google Security Operations should clear the S3 buckets after reading logs, to optimize AWS S3 storage costs. Click Next:Tags.
As a recommended alternative to the previous step, you can further restrict access to only the specified S3 bucket by creating a custom policy. Click Create policy and follow the AWS documentation to create a custom policy.
Add any tags if required, and click Next:Review.
Review the configuration and click Create user.
Copy the Access key ID and Secret access key of the created user, for use in the next step.

Configure Feed in Google Security Operations to Ingest AWS Logs

Go to Google Security Operations settings, and click Feeds.
Click Add New.
Select either Amazon SQS or Amazon S3 as the feed Source Type.
Select AWS CloudTrail (or other AWS service) for Log Type.

Click Next.
Select region and provide S3 URI of the Amazon S3 bucket you copied earlier. Further you could append the S3 URI with:
```
{{datetime("yyyy/MM/dd")}}
```
As in the following example, so that Google Security Operations would scan logs each time only for a particular day:
```
s3://aws-cloudtrail-logs-XXX-1234567/AWSLogs/1234567890/CloudTrail/us-east-1/{{datetime("yyyy/MM/dd")}}/
```
Under URI IS A select Directories including subdirectories. Select an appropriate option under Source Deletion Option, this should match with the permissions of the IAM User account we created earlier.
Provide Access Key ID and Secret Access Key of the IAM User account we created earlier.
Click Next and Finish.

Steps to ingest AWS context data

To ingest context data about AWS entities (such as hosts, instances, and users) create a feed for each of the following log types, listed by description and ingestion label:

AWS EC2 HOSTS (AWS_EC2_HOSTS)
AWS EC2 INSTANCES (AWS_EC2_INSTANCES)
AWS EC2 VPCS (AWS_EC2_VPCS)
AWS Identity and Access Management (IAM) (AWS_IAM)

To create a feed for each of these log types, as follows:

In the navigation bar, select Settings, SIEM Settings, and then Feeds.
On the Feeds page, click Add New. The Add feed dialog appears.
In the Source type menu, select Third party API.
In the Log Type menu, select AWS EC2 Hosts.
Click Next.
Enter the input parameters for the feed in the fields.
Click Next, and then Finish.

For more detailed information about setting up a feed for each log type, see the following Feed management documentation:

For general information about creating a feed, see Feed management user guide or Feed management API.

Field mapping reference

This parser code processes AWS CloudTrail logs in JSON format. It first extracts and structures the raw log message, then iterates through each record in the "Records" array, normalizing single events into the same format as multi-events. Finally, it maps the extracted fields to the Google Security Operations UDM schema, enriching the data with additional context and security-relevant information.

UDM Mapping Table

Log field	UDM mapping	Logic
Records.0.additionalEventData .AuthenticationMethod	additional.fields .AuthenticationMethod.value.string_value	Direct mapping from the raw log field.
Records.0.additionalEventData .CipherSuite	additional.fields .CipherSuite.value.string_value	Direct mapping from the raw log field.
Records.0.additionalEventData .LoginTo	additional.fields .LoginTo.value.string_value	Direct mapping from the raw log field.
Records.0.additionalEventData .MFAUsed	extensions.auth.auth_details	If the value is "Yes", the UDM field is set to "MFAUsed: Yes". Otherwise, it is set to "MFAUsed: No".
Records.0.additionalEventData .MobileVersion	additional.fields .MobileVersion.value.string_value	Direct mapping from the raw log field.
Records.0.additionalEventData .SamlProviderArn	additional.fields .SamlProviderArn.value.string_value	Direct mapping from the raw log field.
Records.0.additionalEventData .SignatureVersion	additional.fields .SignatureVersion.value.string_value	Direct mapping from the raw log field.
Records.0.additionalEventData .bytesTransferredIn	network.received_bytes	Direct mapping from the raw log field, converted to an unsigned integer.
Records.0.additionalEventData .bytesTransferredOut	network.sent_bytes	Direct mapping from the raw log field, converted to an unsigned integer.
Records.0.additionalEventData .x-amz-id-2	additional.fields .x-amz-id-2.value.string_value	Direct mapping from the raw log field.
Records.0.awsRegion	principal.location.name	Direct mapping from the raw log field.
Records.0.awsRegion	target.location.name	Direct mapping from the raw log field.
Records.0.errorCode	security_result.rule_id	Direct mapping from the raw log field.
Records.0.errorMessage	security_result.description	The UDM field is set to "Reason: " concatenated with the value from the raw log field.
Records.0.eventCategory	security_result.category_details	Direct mapping from the raw log field.
Records.0.eventID	metadata.product_log_id	Direct mapping from the raw log field.
Records.0.eventName	metadata.product_event_type	Direct mapping from the raw log field.
Records.0.eventName	_metadata.event_type	Mapped based on the value of the raw log field. See parser code for specific mappings.
Records.0.eventSource	target.application	Direct mapping from the raw log field.
Records.0.eventSource	metadata.ingestion_labels.EventSource	Direct mapping from the raw log field.
Records.0.eventTime	metadata.event_timestamp	Direct mapping from the raw log field, parsed as an ISO8601 timestamp.
Records.0.eventVersion	metadata.product_version	Direct mapping from the raw log field.
Records.0.managementEvent	additional.fields.ManagementEvent .value.string_value	Direct mapping from the raw log field, converted to a string.
Records.0.readOnly	additional.fields.ReadOnly .value.string_value	Direct mapping from the raw log field, converted to a string.
Records.0.recipientAccountId	principal.user.group_identifiers	Direct mapping from the raw log field.
Records.0.recipientAccountId	target.resource.attribute .labels.Recipient Account Id.value	Direct mapping from the raw log field.
Records.0.requestID	target.resource.attribute .labels.Request ID.value	Direct mapping from the raw log field.
Records.0.requestParameters	target.resource.attribute .labels	Various fields within requestParameters are mapped to labels within the target resource attribute. See parser code for specific mappings.
Records.0.requestParameters> .AccessControlPolicy.AccessControlList .Grant.0.Grantee.URI	target.resource.attribute .labels.AccessControlList Grantee URI.value	Direct mapping from the raw log field.
Records.0.requestParameters .AccessControlPolicy.AccessControlList .Grant.1.Grantee.URI	target.resource.attribute .labels.AccessControlList Grantee URI.value	Direct mapping from the raw log field.
Records.0.requestParameters .AccessControlPolicy.AccessControlList .Grant.2.Grantee.URI	target.resource.attribute .labels.AccessControlList Grantee URI.value	Direct mapping from the raw log field.
Records.0.requestParameters .AccessControlPolicy.AccessControlList .Grant.3.Grantee.URI	target.resource.attribute .labels.AccessControlList Grantee URI.value	Direct mapping from the raw log field.
Records.0.requestParameters .AccessControlPolicy.AccessControlList .Grant.4.Grantee.URI	target.resource.attribute .labels.AccessControlList Grantee URI.value	Direct mapping from the raw log field.
Records.0.requestParameters .CreateAccessPointRequest. PublicAccessBlockConfiguration.BlockPublicAcls	target.resource.attribute .labels.BlockPublicAcls.value	Direct mapping from the raw log field, converted to a string.
Records.0.requestParameters .CreateAccessPointRequest. PublicAccessBlockConfiguration.BlockPublicPolicy	target.resource.attribute .labels.BlockPublicPolicy.value	Direct mapping from the raw log field, converted to a string.
Records.0.requestParameters .CreateAccessPointRequest. PublicAccessBlockConfiguration.IgnorePublicAcls	target.resource.attribute .labels.IgnorePublicAcls.value	Direct mapping from the raw log field, converted to a string.
Records.0.requestParameters .CreateAccessPointRequest. PublicAccessBlockConfiguration.RestrictPublicBuckets	target.resource.attribute .labels.RestrictPublicBuckets.value	Direct mapping from the raw log field, converted to a string.
Records.0.requestParameters .PublicAccessBlockConfiguration.BlockPublicAcls	target.resource.attribute .labels.BlockPublicAcls.value	Direct mapping from the raw log field, converted to a string.
Records.0.requestParameters .PublicAccessBlockConfiguration.BlockPublicPolicy	target.resource.attribute .labels.BlockPublicPolicy.value	Direct mapping from the raw log field, converted to a string.
Records.0.requestParameters .PublicAccessBlockConfiguration.IgnorePublicAcls	target.resource.attribute .labels.IgnorePublicAcls.value	Direct mapping from the raw log field, converted to a string.
Records.0.requestParameters .PublicAccessBlockConfiguration.RestrictPublicBuckets	target.resource.attribute .labels.RestrictPublicBuckets.value	Direct mapping from the raw log field, converted to a string.
Records.0.requestParameters.accessKeyId	target.resource.product_object_id	Direct mapping from the raw log field.
Records.0.requestParameters.allocationId	target.resource.product_object_id	Direct mapping from the raw log field.
Records.0.requestParameters.associationId	target.resource.attribute .labels.requestParameters associationId.value	Direct mapping from the raw log field.
Records.0.requestParameters.certificateId	target.resource.product_object_id	Direct mapping from the raw log field.
Records.0.requestParameters .configurationRecorder.name	target.resource.name	Direct mapping from the raw log field.
Records.0.requestParameters .configurationRecorderName	target.resource.name	Direct mapping from the raw log field.
Records.0.requestParameters .createVolumePermission.add.items.0.group	target.resource.attribute .labels.Add Items Group.value	Direct mapping from the raw log field.
Records.0.requestParameters .createVolumePermission.add.items.0.userId	target.resource.attribute .labels.Add Items UserId.value	Direct mapping from the raw log field.
Records.0.requestParameters .createVolumePermission.remove.items.0.userId	target.resource.attribute .labels.Remove Items UserId.value	Direct mapping from the raw log field.
Records.0.requestParameters.detectorId	target.resource.product_object_id	Direct mapping from the raw log field.
Records.0.requestParameters.destinationId	target.resource.product_object_id	Direct mapping from the raw log field.
Records.0.requestParameters.directoryId	target.resource.product_object_id	Direct mapping from the raw log field.
Records.0.requestParameters.documentName	target.resource.product_object_id	Direct mapping from the raw log field.
Records.0.requestParameters.egress	target.resource.attribute .labels.requestParameters egress.value	Direct mapping from the raw log field.
Records.0.requestParameters.emailIdentity	target.resource.name	Direct mapping from the raw log field.
Records.0.requestParameters.enabled	target.resource.attribute .labels.Request Enabled.value	Direct mapping from the raw log field, converted to a string.
Records.0.requestParameters .filterSet.items.0 .valueSet.items.0.value	target.resource.attribute .labels.requestParameters .filterSet.items.0.valueSet .items.0.value.value	Direct mapping from the raw log field.
Records.0.requestParameters.functionName	target.resource.name	Direct mapping from the raw log field.
Records.0.requestParameters .granteePrincipal	principal.hostname	Direct mapping from the raw log field.
Records.0.requestParameters .granteePrincipal	principal.asset.hostname	Direct mapping from the raw log field.
Records.0.requestParameters.groupId	target.resource.product_object_id	Direct mapping from the raw log field.
Records.0.requestParameters.groupName	target.group.group_display_name	Direct mapping from the raw log field.
Records.0.requestParameters.imageId	target.resource.product_object_id	Direct mapping from the raw log field.
Records.0.requestParameters.instanceId	target.resource_ancestors.product_object_id	Direct mapping from the raw log field.
Records.0.requestParameters .instanceProfileName	target.resource.name	Direct mapping from the raw log field.
Records.0.requestParameters.instanceType	target.resource.attribute .labels.Instance Type.value	Direct mapping from the raw log field.
Records.0.requestParameters .instancesSet.items.0.instanceId	target.resource.product_object_id	Direct mapping from the raw log field.
Records.0.requestParameters .instancesSet.items.0.maxCount	target.resource.attribute .labels.Instance Set Max Count.value	Direct mapping from the raw log field, converted to a string.
Records.0.requestParameters .instancesSet.items.0.minCount	target.resource.attribute .labels.Instance Set Min Count.value	Direct mapping from the raw log field, converted to a string.
Records.0.requestParameters .ipPermissions.items.0 .ipRanges.items.0.cidrIp	target.resource.attribute .labels.ipPermissions cidrIp.value	Direct mapping from the raw log field.
Records.0.requestParameters .ipPermissions.items.0 .ipv6Ranges.items.0.cidrIpv6	target.resource.attribute .labels.ipPermissions cidrIpv6.value	Direct mapping from the raw log field.
Records.0.requestParameters .ipPermissions.items.1 .ipv6Ranges.items.0.cidrIpv6	target.resource.attribute .labels.ipPermissions cidrIpv6.value	Direct mapping from the raw log field.
Records.0.requestParameters.keyId	target.resource.product_object_id	Direct mapping from the raw log field.
Records.0.requestParameters. launchPermission.add.items.0.group	target.resource.attribute .labels.Add Items Group.value	Direct mapping from the raw log field.
Records.0.requestParameters. launchPermission.add.items .0.organizationalUnitArn	target.resource.attribute.labels .Add Items OrganizationalUnitArn .value	Direct mapping from the raw log field.
Records.0.requestParameters. launchPermission.add.items .0.userId	target.resource.attribute .labels.Add Items UserId.value	Direct mapping from the raw log field.
Records.0.requestParameters. launchPermission.remove.items .0.organizationalUnitArn	target.resource.attribute.labels .Remove Items OrganizationalUnitArn .value	Direct mapping from the raw log field.
Records.0.requestParameters. launchPermission.remove.items .0.userId	target.resource.attribute .labels.Remove Items UserId.value	Direct mapping from the raw log field.
Records.0.requestParameters.loadBalancerArn	target.resource.name	Direct mapping from the raw log field.
Records.0.requestParameters.logGroupIdentifier	target.resource.name	Direct mapping from the raw log field.
Records.0.requestParameters.logGroupName	target.resource.name	Direct mapping from the raw log field.
Records.0.requestParameters.name	target.resource.name	Direct mapping from the raw log field.
Records.0.requestParameters.name	target.resource.product_object_id	Direct mapping from the raw log field.
Records.0.requestParameters.networkAclId	target.resource.product_object_id	Direct mapping from the raw log field.
Records.0.requestParameters .networkInterfaceId	target.resource.product_object_id	Direct mapping from the raw log field.
Records.0.requestParameters.parentId	target.resource_ancestors.product_object_id	Direct mapping from the raw log field.
Records.0.requestParameters.policyArn	target.resource.name	Direct mapping from the raw log field.
Records.0.requestParameters .policyArns.0.arn	target.resource.attribute .labels.Policy ARN 0.value	Direct mapping from the raw log field.
Records.0.requestParameters .policyArns.1.arn	target.resource.attribute .labels.Policy ARN 1.value	Direct mapping from the raw log field.
Records.0.requestParameters.policyName	target.resource.attribute .permissions.name	Direct mapping from the raw log field.
Records.0.requestParameters.policyName	target.resource.name	Direct mapping from the raw log field.
Records.0.requestParameters.principalArn	principal.resource.name	Direct mapping from the raw log field.
Records.0.requestParameters.publicKeyId	target.resource.product_object_id	Direct mapping from the raw log field.
Records.0.requestParameters.RegionName	target.resource.name	Direct mapping from the raw log field.
Records.0.requestParameters.RegionName	target.resource.name	Direct mapping from the raw log field.
Records.0.requestParameters.roleName	target.resource.name	Direct mapping from the raw log field.
Records.0.requestParameters.sAMLProviderArn	target.resource.name	Direct mapping from the raw log field.
Records.0.requestParameters.secretId	target.resource.product_object_id	Direct mapping from the raw log field.
Records.0.requestParameters.serialNumber	target.resource.name	Direct mapping from the raw log field.
Records.0.requestParameters .serviceSpecificCredentialId	target.resource.product_object_id	Direct mapping from the raw log field.
Records.0.requestParameters.sendingEnabled	target.resource.attribute .labels.Request Sending Enabled.value	Direct mapping from the raw log field, converted to a string.
Records.0.requestParameters.snapshotId	target.resource.product_object_id	Direct mapping from the raw log field.
Records.0.requestParameters.sSHPublicKeyId	target.resource.product_object_id	Direct mapping from the raw log field.
Records.0.requestParameters.stackName	target.resource.product_object_id	Direct mapping from the raw log field.
Records.0.requestParameters.status	target.resource.attribute .labels.Request Parameter Status.value	Direct mapping from the raw log field.
Records.0.requestParameters.subnetId	target.resource.attribute .labels.Subnet Id.value	Direct mapping from the raw log field.
Records.0.requestParameters .targets.0.InstanceIds	target.resource.attribute .labels.requestParameters.targets .0.InstanceIds.value	Direct mapping from the raw log field.
Records.0.requestParameters .targets.0.key	target.resource.attribute .labels.requestParameters.targets.0.key.value	Direct mapping from the raw log field.
Records.0.requestParameters.trailName	target.resource.name	Direct mapping from the raw log field.
Records.0.requestParameters.userName	target.user.userid	Direct mapping from the raw log field.
Records.0.requestParameters.volumeId	target.resource.product_object_id	Direct mapping from the raw log field.
Records.0.requestParameters.withDecryption	security_result.detection_fields .withDecryption.value	Direct mapping from the raw log field, converted to a string.
Records.0.responseElements	target.resource.attribute.labels	Various fields within responseElements are mapped to labels within the target resource attribute. See parser code for specific mappings.
Records.0.responseElements.accessKey.accessKeyId	target.resource.product_object_id	Direct mapping from the raw log field.
Records.0.responseElements.accessKey.status	target.resource.attribute .labels.Response Access Key Status.value	Direct mapping from the raw log field.
Records.0.responseElements.accessKey.userName	target.user.userid	Direct mapping from the raw log field.
Records.0.responseElements.allocationId	target.resource.product_object_id	Direct mapping from the raw log field.
Records.0.responseElements .certificate.certificateId	target.resource.product_object_id	Direct mapping from the raw log field.
Records.0.responseElements .certificate.status	target.resource.attribute .labels.Certificate Status.value	Direct mapping from the raw log field.
Records.0.responseElements .certificate.userName	target.user.userid	Direct mapping from the raw log field.
Records.0.responseElements .credentials.accessKeyId	target.resource.product_object_id	Direct mapping from the raw log field.
Records.0.responseElements .credentials.sessionToken	security_result.detection_fields .sessionToken.value	Direct mapping from the raw log field.
Records.0.responseElements .createAccountStatus.accountId	target.resource.name	Direct mapping from the raw log field.
Records.0.responseElements .createAccountStatus.accountName	target.user.user_display_name	Direct mapping from the raw log field.
Records.0.responseElements .createAccountStatus.accountName	target.user.user_display_name	Direct mapping from the raw log field.
Records.0.responseElements .createAccountStatus.accountName	target.user.user_display_name	Direct mapping from the raw log field.
Records.0.responseElements .createCollectionDetail.arn	target.resource.name	Direct mapping from the raw log field.
Records.0.responseElements .createCollectionDetail.id	target.resource.product_object_id	Direct mapping from the raw log field.
Records.0.responseElements .deleteCollectionDetail.id	target.resource.product_object_id	Direct mapping from the raw log field.
Records.0.responseElements.description	target.resource.attribute .labels.Response Elements Description.value	Direct mapping from the raw log field.
Records.0.responseElements.destinationId	target.resource.product_object_id	Direct mapping from the raw log field.
Records.0.responseElements.detectorId	target.resource.product_object_id	Direct mapping from the raw log field.
Records.0.responseElements.directoryId	target.resource.product_object_id	Direct mapping from the raw log field.
Records.0.responseElements .domainStatus.aRN	target.resource.name	Direct mapping from the raw log field.
Records.0.responseElements .domainStatus.domainId	target.resource.product_object_id	Direct mapping from the raw log field.
Records.0.responseElements .federatedUser.arn	target.resource.name	Direct mapping from the raw log field.
Records.0.responseElements .federatedUser.federatedUserId	target.user.userid	Direct mapping from the raw log field.
Records.0.responseElements .firewall.firewallArn	target.resource.name	Direct mapping from the raw log field.
Records.0.responseElements .firewall.firewallId	target.resource.product_object_id	Direct mapping from the raw log field.
Records.0.responseElements .firewall.firewallName	target.resource.attribute .labels.Firewall Name.value	Direct mapping from the raw log field.
Records.0.responseElements .flowLogIdSet.item	target.resource.product_object_id	Direct mapping from the raw log field.
Records.0.responseElements.functionArn	target.resource.name	Direct mapping from the raw log field.
Records.0.responseElements .group.arn	target.group.product_object_id	Direct mapping from the raw log field.
Records.0.responseElements .group.groupName	target.group.group_display_name	Direct mapping from the raw log field.
Records.0.responseElements .iamInstanceProfileAssociation.instanceId	target.resource.name	Direct mapping from the raw log field.
Records.0.responseElements .iamInstanceProfileAssociation.instanceId	target.resource.product_object_id	Direct mapping from the raw log field.
Records.0.responseElements .image.imageId.imageDigest	src.file.sha256	The UDM field is set to the value after "sha256:" in the raw log field.
Records.0.responseElements .image.imageManifestMediaType	src.file.mime_type	Direct mapping from the raw log field.
Records.0.responseElements.instanceArn	target.resource.name	Direct mapping from the raw log field.
Records.0.responseElements .instanceProfile.arn	target.resource.name	Direct mapping from the raw log field.
Records.0.responseElements .instancesSet.items.0.instanceId	target.resource.product_object_id	Direct mapping from the raw log field.
Records.0.responseElements.keyId	target.resource.name	Direct mapping from the raw log field.
Records.0.responseElements .keyMetadata.arn	target.resource.name	Direct mapping from the raw log field.
Records.0.responseElements .keyMetadata.encryptionAlgorithms	security_result.detection_fields .encryptionAlgorithm.value	The UDM field is set to the value of each element in the array from the raw log field.
Records.0.responseElements .keyMetadata.keyId	target.resource.product_object_id	Direct mapping from the raw log field.
Records.0.responseElements.keyPairId	target.resource.product_object_id	Direct mapping from the raw log field.
Records.0.responseElements .listeners.0.listenerArn	target.resource.name	Direct mapping from the raw log field.
Records.0.responseElements .listeners.0.loadBalancerArn	target.resource.ancestors.name	Direct mapping from the raw log field.
Records.0.responseElements .loadBalancers.0.loadBalancerArn	target.resource.name	Direct mapping from the raw log field.
Records.0.responseElements.newAssociationId	target.resource.attribute.labels .responseElements newAssociationId.value	Direct mapping from the raw log field.
Records.0.responseElements.packedPolicySize	security_result.detection_fields .packedPolicySize.value	Direct mapping from the raw log field, converted to a string.
Records.0.responseElements .publicKey.publicKeyId	target.resource.product_object_id	Direct mapping from the raw log field.
Records.0.responseElements.sAMLProviderArn	target.resource.name	Direct mapping from the raw log field.
Records.0.responseElements .sSHPublicKey.sSHPublicKeyId	target.resource.product_object_id	Direct mapping from the raw log field.
Records.0.responseElements .sSHPublicKey.status	target.resource.attribute .labels.SSH Public Key Status.value	Direct mapping from the raw log field.
Records.0.responseElements .securityGroupRuleSet.items.0.groupId	security_result.rule_labels.Group Id.value	Direct mapping from the raw log field.
Records.0.responseElements .securityGroupRuleSet.items.0.ipProtocol	network.ip_protocol	Direct mapping from the raw log field, converted to uppercase.
Records.0.responseElements .securityGroupRuleSet.items.0.isEgress	network.direction	If the value is "false", the UDM field is set to "INBOUND". Otherwise, it is set to "OUTBOUND".
Records.0.responseElements .securityGroupRuleSet.items.0.securityGroupRuleId	security_result.rule_id	Direct mapping from the raw log field.
Records.0.responseElements .serviceSpecificCredential.serviceName	target.resource.attribute.labels .Specific Credential ServiceName .value	Direct mapping from the raw log field.
Records.0.responseElements .serviceSpecificCredential.serviceSpecificCredentialId	target.resource.product_object_id	Direct mapping from the raw log field.
Records.0.responseElements .serviceSpecificCredential.serviceUserName	target.resource.attribute.labels .Specific Credential Service UserName .value	Direct mapping from the raw log field.
Records.0.responseElements .serviceSpecificCredential.status	target.resource.attribute .labels.Specific Credential Status.value	Direct mapping from the raw log field.
Records.0.responseElements .serviceSpecificCredential.userName	target.user.userid	Direct mapping from the raw log field.
Records.0.responseElements.snapshotId	target.resource.product_object_id	Direct mapping from the raw log field.
Records.0.responseElements.stackId	target.resource.name	Direct mapping from the raw log field.
Records.0.responseElements .tableDescription.tableArn	target.resource.name	Direct mapping from the raw log field.
Records.0.responseElements .tableDescription.tableId	target.resource.product_object_id	Direct mapping from the raw log field.
Records.0.responseElements.trailARN	target.resource.name	Direct mapping from the raw log field.
Records.0.responseElements .user.arn	target.user.userid	Direct mapping from the raw log field.
Records.0.responseElements .user.userId	target.user.product_object_id	Direct mapping from the raw log field.
Records.0.responseElements .user.userName	target.user.user_display_name	Direct mapping from the raw log field.
Records.0.responseElements .virtualMFADevice.serialNumber	target.resource.name	Direct mapping from the raw log field.
Records.0.responseElements.volumeId	target.resource.product_object_id	Direct mapping from the raw log field.
Records.0.resources	target.resource	The first element in the resources array is mapped to the target resource. Other elements are mapped to the about field.
Records.0.sharedEventID	additional.fields.SharedEventID .value.string_value	Direct mapping from the raw log field.
Records.0.sourceIPAddress	principal.asset.ip	Direct mapping from the raw log field.
Records.0.sourceIPAddress	principal.ip	Direct mapping from the raw log field.
Records.0.sourceIPAddress	src_ip	Direct mapping from the raw log field.
Records.0.tlsDetails.cipherSuite	network.tls.cipher	Direct mapping from the raw log field.
Records.0.tlsDetails.clientProvidedHostHeader	security_result.detection_fields .clientProvidedHostHeader.value	Direct mapping from the raw log field.
Records.0.tlsDetails.tlsVersion	network.tls.version	Direct mapping from the raw log field.
Records.0.userAgent	network.http.user_agent	Direct mapping from the raw log field.
Records.0.userAgent	network.http.parsed_user_agent	Direct mapping from the raw log field, parsed as a user agent string.
Records.0.userIdentity.accessKeyId	additional.fields.accessKeyId .value.string_value	Direct mapping from the raw log field.
Records.0.userIdentity.accountId	principal.resource.product_object_id	Direct mapping from the raw log field.
Records.0.userIdentity.accountId	principal.user.group_identifiers	Direct mapping from the raw log field.
Records.0.userIdentity.arn	principal.resource.name	Direct mapping from the raw log field.
Records.0.userIdentity.arn	principal.user.userid	Direct mapping from the raw log field.
Records.0.userIdentity.arn	target.user.attribute .labels.ARN.value	Direct mapping from the raw log field.
Records.0.userIdentity.invokedBy	principal.user.userid	The UDM field is set to the value before ".amazonaws.com" in the raw log field.
Records.0.userIdentity.principalId	principal.user.product_object_id	Direct mapping from the raw log field.
Records.0.userIdentity.principalId	principal.user.attribute .labels.principalId.value	Direct mapping from the raw log field.
Records.0.userIdentity .sessionContext.attributes.mfaAuthenticated	principal.user.attribute .labels.mfaAuthenticated.value	Direct mapping from the raw log field.
Records.0.userIdentity .sessionContext.sessionIssuer.arn	target.user.attribute .labels.ARN.value	Direct mapping from the raw log field.
Records.0.userIdentity .sessionContext.sessionIssuer.principalId	target.user.userid	Direct mapping from the raw log field.
Records.0.userIdentity .sessionContext.sessionIssuer.type	target.user.attribute .labels.Type.value	Direct mapping from the raw log field.
Records.0.userIdentity .sessionContext.sessionIssuer.userName	target.user.user_display_name	Direct mapping from the raw log field.
Records.0.userIdentity.type	principal.resource.resource_subtype	Direct mapping from the raw log field.
Records.0.userIdentity.type	principal.resource.type	Direct mapping from the raw log field.
Records.0.userIdentity.userName	principal.user.user_display_name	Direct mapping from the raw log field.
Records.0.userIdentity.userName	src.user.userid	Direct mapping from the raw log field.
Records.0.userIdentity.userName	src.user.user_display_name	Direct mapping from the raw log field.
Records.0.userIdentity.userName	target.user.user_display_name	Direct mapping from the raw log field.
Records.1.additionalEventData .AuthenticationMethod	additional.fields.AuthenticationMethod .value.string_value	Direct mapping from the raw log field.
Records.1.additionalEventData .CipherSuite	additional.fields.CipherSuite .value.string_value	Direct mapping from the raw log field.
Records.1.additionalEventData .LoginTo	additional.fields.LoginTo .value.string_value	Direct mapping from the raw log field.
Records.1.additionalEventData .MFAUsed	extensions.auth.auth_details	If the value is "Yes", the UDM field is set to "MFAUsed: Yes". Otherwise, it is set to "MFAUsed: No".
Records.1.additionalEventData .MobileVersion	additional.fields.MobileVersion .value.string_value	Direct mapping from the raw log field.
Records.1.additionalEventData .SamlProviderArn	additional.fields.SamlProviderArn .value.string_value	Direct mapping from the raw log field.
Records.1.additionalEventData .SignatureVersion	additional.fields.SignatureVersion .value.string_value	Direct mapping from the raw log field.
Records.1.additionalEventData .bytesTransferredIn	network.received_bytes	Direct mapping from the raw log field, converted to an unsigned integer.
Records.1.additionalEventData .bytesTransferredOut	network.sent_bytes	Direct mapping from the raw log field, converted to an unsigned integer.
Records.1.additionalEventData .x-amz-id-2	additional.fields.x-amz-id-2 .value.string_value	Direct mapping from the raw log field.
Records.1.awsRegion	principal.location.name	Direct mapping from the raw log field.
Records.1.awsRegion	target.location.name	Direct mapping from the raw log field.
Records.1.errorCode	security_result.rule_id	Direct mapping from the raw log field.
Records.1.errorMessage	security_result.description	The UDM field is set to "Reason: " concatenated with the value from the raw log field.
Records.1.eventCategory	security_result.category_details	Direct mapping from the raw log field.
Records.1.eventID	metadata.product_log_id	Direct mapping from the raw log field.
Records.1.eventName	metadata.product_event_type	Direct mapping from the raw log field.
Records.1.eventName	_metadata.event_type	Mapped based on the value of the raw log field. See parser code for specific mappings.
Records.1.eventSource	target.application	Direct mapping from the raw log field.
Records.1.eventSource	metadata.ingestion_labels.EventSource	Direct mapping from the raw log field.
Records.1.eventTime	metadata.event_timestamp	Direct mapping from the raw log field, parsed as an ISO8601 timestamp.
Records.1.eventVersion	metadata.product_version	Direct mapping from the raw log field.
Records.1.managementEvent	additional.fields.ManagementEvent .value.string_value	Direct mapping from the raw log field, converted to a string.
Records.1.readOnly	additional.fields.ReadOnly .value

Changes

2024-07-30

Fixed the mapping of "src_ip" and "event_type" to parse the new logs.

2024-07-29

Bug-Fix:
When "eventName" is "GetLoginProfile" then mapped "metadata.event_type" to "RESOURCE_READ".

2024-07-24

Changed the mapping from "recipientAccountId" to "userIdentity.accountId" and mapped it to "additional.fields".

2024-07-23

Mapped "alert_emails" and "owner_names" to "target.resource.attribute.labels".

2024-07-09

Mapped "eventVersion" to "metadata.product_version".
Mapped "userIdentity.principalId" to "principal.user.attribute.labels".
Mapped "userIdentity.sessionContext.attributes.creationDate" to "principal.user.attribute.creation_time".
Mapped "userIdentity.sessionContext.sessionIssuer.type" to "target.user.attribute.labels".
Mapped "additionalEventData.bytesTransferredIn" to "network.received_bytes".
Mapped "additionalEventData.bytesTransferredOut" to "network.sent_bytes".
Mapped "managementEvent", "readOnly", "sharedEventID", "apiVersion", "additionalEventData.x-amz-id-2", "additionalEventData.SignatureVersion", "additionalEventData.AuthenticationMethod", "additionalEventData.CipherSuite", and "additionalEventData.sub" to "additional.fields".

2024-06-24

Added support for a new pattern of JSON logs.

2024-06-24

Updated the mapping from "principal.resource.type" to "principal.resource.resource_subtype" since the field "principal.resource.type" is a deprecated field.

2024-05-21

When "requestParameters.bucketPolicy.Statement.n.Resource" is an array, then mapped "requestParameters.bucketPolicy.Statement.n.Resource" to "additional.fields".

2024-05-09

Mapped the "groupid" part from "principal.user.userid" to "principal.user.groupid" and "principal.user.group_identifiers" when the "userid" matches the format "^arn:aws:sts::\d+:assumed-role\/\w+\/\w+$".

2024-04-30

Mapped "req.requestParameters.networkInterfaceSet.items.associatePublicIpAddress" to "target.resource.attribute.labels".

2024-03-22

Mapped "Noun.user.userid" to "Noun.user.product_object_id".
Mapped "RoleName" from "userIdentity.arn" to "principal.user.role_name" and "principal.user.attribute.roles.name".
Mapped "PoicyName" from "requestParameters.policyArn" to "security_result.rule_name".

2024-03-04

For logs having "eventName" as "TerminateInstances":
Mapped "responseElements" JSON Object to "target.resource.attribute.labels".
Mapped "sessionCredentialFromConsole" to "target.resource.attribute.labels".
For logs where "eventName" is "CreateDomain","DeleteDomain","CreateCollection",
"DeleteCollection","CreateDBCluster","DeleteDBCluster","StopDBCluster","StartDBCluster",
"CreateCluster","DeleteCluster", "ListClusters", "CreateNodegroup", "DeleteNodegroup",
"RegisterCluster", "DeregisterCluster", "DescribeCluster", "DescribeNodegroup", "ListNodegroups".
Set "target.resource.resource_type" to "CLUSTER".

2023-11-21

Mapped "awsRegion" to "target.location.name".
For logs having "eventName" as "PutBucketAcl", when "userIdentity.arn" is not present, then modify "metadata.event_type" to "STATUS_UPDATE".
For logs having "eventName" as prefix "Get", "List", "Describe", "Detect", "Query", "Check", "Decode",
"Decrypt", "Download", "Retrieve", "Read", "Discover", "Lookup", "Preview", "Scan", "Select", "Classify", "Show", "View":
Set "metadata.event_type" to "RESOURCE_READ".
For logs having "eventName" as prefix "Delete", "Terminate":
Set "metadata.event_type" to "RESOURCE_DELETION".
For logs having "eventName" as prefix "Create", "Put", "Import", "Generate", "Allocate":
Set "metadata.event_type" to "RESOURCE_CREATION".
For logs having "eventName" as prefix "Start", "Activate", "Reboot", "Initialize", "New":
Set "metadata.event_type" to "STATUS_STARTUP".
For logs having "eventName" as prefix "Stop", "Cancel", "Disconnect":
Set "metadata.event_type" to "STATUS_SHUTDOWN".
For logs having "eventName" as prefix "Test", "Accept", "Notify", "Request", "Validate", "Confirm", "Reject", "Verify", "Authorize", "Complete":
Set "metadata.event_type" to "STATUS_UPDATE".
For logs having "eventName" as prefix "Assume", "ConsoleLogin":
Set "metadata.event_type" to "USER_LOGIN".
For logs having "eventName" as "SendHeartbeat":
Set "metadata.event_type" to "STATUS_HEARTBEAT".
For logs haveing "eventName" as prefix "Initiate", "Publish", "Replace", "Resume", "Run", "Submit", "Suspend",
"Alter", "Increase", "Invite", "Provision", "Refresh", "Report", "Upgrade", "Abort", "Apply", "Backup", "Decrease",
"Merge", "Retry", "Rotate", "Rotation", "Transfer", "Unassign", "Analyze", "Archive", "Beta_", "Clear", "Configure",
"Confirm_", "Do", "Evaluate", "Failover", "Forgot", "Lock", "Migrate", "O", "Process", "Promote", "Release", "Renew",
"Sign", "Unarchive", "Undeprecate", "Unlock", "Acknowledge", "Approve", "Connect", "Continue", "Decline", "Deploy",
"Diagnostic", "Drop", "Exit", "Finalize", "Flush", "Forget", "Grant", "Issue", "Logout", "Move", "Opt", "Pause",
"Rebuild", "Redeem", "Replicate", "Restart", "S", "Save", "Subscribe", "Sync", "Unlink", "Unsubscribe", "Unsuspend",
"Allow", "Ato", "Back", "Backtrack", "Bid", "Bind", "Build", "Bundle", "Clone", "Close", "Cognito", "Console", "Dispose",
"Dissociate", "End", "Enroll", "Enter", "Environment", "Event_", "Exclude", "Global", "Include", "Index", "Insert", "Install",
"Invalidate", "Join", "Leave", "Load", "Managed", "Mark", "Monitor", "Peer", "Persist", "Prepare", "Pubkey", "Purge", "Push",
"Rebalance", "Record", "Recovery", "Redact", "Refuse", "Reinvite", "Reload", "Rename", "Respond", "Resync", "Retire", "Reverse",
"Rollback", "Schedule", "Secret", "Shutdown", "Signal", "Skip", "Split", "Stream", "Swap", "Switch", "Toggle", "Token_",
"Translate", "Trim", "Unauthorize", "Undeploy", "Unmonitor", "Unpeer", "Use":
Set "metadata.event_type" to "RESOURCE_WRITTEN".
For logs haveing "eventName" as prefix "Update", "Associate", "Disassociate", "Modify", "Set", "Register", "Deregister",
"Add", "Remove", "Enable", "Disable", "Send", "Restore", "Reset", "Attach", "Detach", "Export", "Copy", "Tag",
"Untag", "Execute", "Purchase", "Allocate", "Deactivate", "Post", "Resend", "Upload", "Assign", "Change", "Define",
"Deprecate", "Invoke", "Revoke:
Set "metadata.event_type" to "RESOURCE_PERMISSIONS_CHANGE".

2023-11-11

Initialize variables to null or empty, to avoid duplicate mappings.
When "requestParameters.tagSpecificationSet.items.key" is "Hostname" , map to "target.hostname".

2023-10-27

For logs having "eventName" as "AssociateIamInstanceProfile":
Mapped "responseElements.AssociateIamInstanceProfileResponse.iamInstanceProfileAssociation.instanceid" to "target.resource.name".
Mapped "responseElements.AssociateIamInstanceProfileResponse.iamInstanceProfileAssociation.instanceid" to "target.resource.product_object_id".
Set "metadata.event_type" to "RESOURCE_PERMISSIONS_CHANGE".
Set "target.resource.resource_type" to "ACCESS_POLICY".
For logs having "eventName" as "DisassociateIamInstanceProfile":
Mapped "responseElements.DisassociateIamInstanceProfileResponse.iamInstanceProfileAssociation.instanceid" to "target.resource.name".
Mapped "responseElements.DisassociateIamInstanceProfileResponse.iamInstanceProfileAssociation.instanceid" to "target.resource.product_object_id".
Set "metadata.event_type" to "RESOURCE_PERMISSIONS_CHANGE".
Set "target.resource.resource_type" to "ACCESS_POLICY".
For logs having "eventName" as "ReplaceIamInstanceProfileAssociation":
Mapped "responseElements.ReplaceIamInstanceProfileAssociationResponse.iamInstanceProfileAssociation.instanceid" to "target.resource.name".
Mapped "responseElements.ReplaceIamInstanceProfileAssociationResponse.iamInstanceProfileAssociation.instanceid" to "target.resource.product_object_id".
Set "metadata.event_type" to "RESOURCE_PERMISSIONS_CHANGE".
Set "target.resource.resource_type" to "ACCESS_POLICY".
Mapped "requestParameters" and "responseElements" JSON Object to "target.resource.attribute.labels".
Corrected typo error for "req.userIdentity.userName" from "req.userIdentity.username".

2023-10-13

For logs having "eventName" as "UpdateDetector":
Mapped "requestParameters.features.name" and "requestParameters.features.status" to "target.resource.attribute.labels".
For logs having "eventName" as "SendCommand":
Mapped "requestParameters.documentName" to "target.resource.product_object_id".
Mapped "responseElements.command.commandId" to "target.process.product_specific_object.id".
Mapped "metadata.event_type" to "PROCESS_LAUNCH".
Mapped "requestParameters.documentName" to "target.resource.name".
Mapped all the parameters in "requestParameters" and "responseElements" to "target.resource.attribute.labels".
For logs having "eventName" as "createAccountResult" map "event_type" as "USER_RESOURCE_ACCESS".
For logs having "eventName" as "createAccount" map "event_type" as "RESOURCE_CREATION".

2023-09-30

add new mappings for the following fields:
Mapped "req.requestParameters.durationSeconds" to "target.resource.attribute.labels".
Mapped "req.requestParameters.policyArns" to "target.resource.attribute.labels".
For logs having "eventName" as "GetParameter", "GetParameters", "GetParameterHistory", "GetParametersByPath", "DescribeParameters":
Mapped "metadata.event_type" to "RESOURCE_READ".
Mapped "req.requestParameters.withDecryption" to "security_result.detection_fields".
For logs having "eventName" as "DeleteParameters","DeleteParameter", set "metadata.event_type" to "RESOURCE_DELETION".
For logs having "eventName" as "PutParameter", set "metadata.event_type" to "RESOURCE_PERMISSIONS_CHANGE".
For logs having "eventName" as "EnableRegion" or "DisableRegion", set "target.resource.name" from "req.requestParameters.map.RegionName".
For logs having "eventName" as "GetFederationToken":
Mapped "metadata.event_type" to "RESOURCE_READ".
Mapped "req.responseElements.federatedUser.arn" to "target.resource.name".
Mapped "req.responseElements.federatedUser.federatedUserId" to "target.user.userid".
Mapped "req.responseElements.packedPolicySize" to "security_result.detection_fields".
Mapped "req.responseElements.credentials.sessionToken" to "security_result.detection_fields".

2023-09-15

add new mappings for the following fields:
Mapped "requestParameters.userName" to "target.user.user_display_name".
Mapped "additionalEventData.SamlProviderArn" to "additional.fields".
Mapped "eventSource" to "metadata.ingestion_labels".
When value of "requestParameters.tagSpecificationSet.items.tags.key" is "Name", then mapped "requestParameters.tagSpecificationSet.items.tags.value" to "target.resource.name".

2023-08-24

For logs having "eventName" as "CreateFirewall" and "DeleteFirewall" :
Mapped "responseElements.firewallARN" to "target.resource.name".
Mapped "responseElements.firewallId" to "target.resource.product_object_id".
Mapped "responseElements.firewallName" to "target.resource.attribute.labels".
Mapped "target.resource_subtype" as "Firewall".
Mapped "target.resource.resource_type" as "FIREWALL_RULE".

2023-08-24

For logs having "eventName" as "CreateSubnet", set "metadata.event_type" to "RESOURCE_CREATION".
Mapped "req.responseElements.subnet.subnetId" to "target.resource.attribute.labels".
Mapped "req.requestParameters.cidrBlock" to "target.resource.attribute.labels".
For logs having "eventName" as "DeleteSubnet", set "metadata.event_type" to "RESOURCE_DELETION".
Mapped "req.requestParameters.subnetId" to "target.resource.attribute.labels".

2023-08-16

For logs having "eventName" as "DeleteSecret", mapped "responseElements.arn" to "target.resource.name".

2023-08-02

For logs having "eventName" as "CreateTags", mapped "metadata.event_type" to "RESOURCE_WRITTEN".
Mapped "responseElements.description" ,"requestParameters.name","requestParameters.tagSet.items", "requestParameters.attributeType" to "target.resource.attribute.labels".
Set "metadata.event_type" to "RESOURCE_CREATION" for logs having the following "eventName":
"CreateNetworkAcl","CreateVolume","CreatePublishingDestination","CreateIPSet","CreateThreatIntelSet",
"CreateAddon","CreateRepository","CreateStack","CreateDomain","CreateCollection","CreateTable",
"CreateDBInstance","CreateDBCluster","CreateDBSnapshot","CreateDBClusterSnapshot","PutConfigRule",
"PutDeliveryChannel","CreateListener","CreateLoadBalancer","PutLoggingConfiguration","CreateTargetGroup",
"CreateWebACL","RequestCertificate","CreateCluster"
Set "metadata.event_type" to "RESOURCE_WRITTEN for logs having the follow "eventName":
"MoveAccount","PutEventSelectors","PutInsightSelectors","UpdateIPSet","UpdateThreatIntelSet","CreateTags",
"UpdateTable","ModifyDBInstance","StopDBInstance","StartDBInstance","RebootDBInstance",
"StartDBCluster","StopDBCluster","ModifyDBSnapshotAttribute","ModifyDBClusterSnapshotAttribute",
"AddListenerCertificates","ModifyLoadBalancerAttributes","SetSubnets","SetSecurityGroups",
"ModifyListener","UpdateWebACL","ResendValidationEmail","ModifyInstanceAttribute",
"StopInstances","StartInstances","RebootInstances"
Set "metadata.event_type" to "RESOURCE_WRITTEN" for logs having the following "eventName".
"DeletePublishingDestination","DeleteIPSet","DeleteThreatIntelSet","DeleteRepository",
"DeleteStack","DeleteCollection","DeleteDomain","DeleteTable","DeleteDBInstance","DeleteDBCluster",
"DeleteDBSnapshot","DeleteDBClusterSnapshot","DeleteConfigRule","DeleteEvaluationResults",
"DeleteTargetGroup","DeleteLoadBalancer","DeleteListener","DeleteLoggingConfiguration",
"DeleteWebACL","DeleteCertificate","DeleteCluster"
Set "metadata.event_type" to "RESOURCE_PERMISSIONS_CHANGE" for logs having the following "eventName":
"AssociateWebACL","DisassociateWebACL","AttachGroupPolicy","PutBucketAcl"
Set "metadata.event_type" to "RESOURCE_READ" for logs having the following "eventName":
"GetPasswordData","GetSessionToken"
Mapped "target.resource.resource_type" and other unmapped fields for the above mentioned event names.

2023-07-18

For logs with the following "eventName", mapped "metadata.event_type" to "RESOURCE_CREATION".
"EnableMacie","ConnectDirectory","RunInstances","CreateImage","CreateOrganization", "CreateNetworkInterface",
"StartSSO","CreateEmailIdentity","VerifyDomainIdentity","VerifyDomainDkim","VerifyEmailIdentity",
"CreateConfigurationSet","CreateSecret","ImportKeyPair","CreateAlias","CreateKey","CreateOrganizationalUnit",
"CreateNetworkAcl","CreateVolume","CreatePublishingDestination","CreateIPSet","CreateThreatIntelSet"
For logs with the following "eventName", mapped "metadata.event_type" to "RESOURCE_WRITTEN".
"UpdateMacieSession","PutAccountSendingAttributes","PutConfigurationSetSendingOptions","UpdateAccountSendingEnabled",
"UpdateConfigurationSetSendingEnabled","UpdateSecret","DisableKey","EnableKey","CancelKeyDeletion",
"MoveAccount","PutEventSelectors","PutInsightSelectors","UpdateIPSet","UpdateThreatIntelSet"
For logs with the following "eventName", mapped "metadata.event_type" to "RESOURCE_DELETION".
"DeleteSnapshot","DeleteDetector","DeleteFlowLogs","DeregisterImage","TerminateInstances", "RESOURCE_DELETION",
"DeleteNetworkInterface","DeleteSSO","DeleteBucketPublicAccessBlock","DeleteAccountPublicAccessBlock",
"RemoveAccountFromOrganization","DeleteEmailIdentity","LeaveOrganization","DeleteConfigurationSet",
"DeleteSecret","DeleteKeyPair","DeleteAlias","ScheduleKeyDeletion","DeleteNetworkAcl",
"DeletePublishingDestination","DeleteIPSet","DeleteThreatIntelSet"
For logs with the following "eventName", mapped "metadata.event_type" to "RESOURCE_PERMISSIONS_CHANGE".
"DetachRolePolicy","PutRolePolicy","PutResourcePolicy","PutCredentials","DeleteDirectory",
"AuthorizeSecurityGroupEgress","AuthorizeSecurityGroupIngress","RevokeSecurityGroupEgress","RevokeSecurityGroupIngress",
"ModifySnapshotAttribute","ModifyImageAttribute","CreateNetworkAclEntry","ReplaceNetworkAclAssociation","DeleteNetworkAclEntry"
Mapped "target.resource.resource_type" and other unmapped fields for the above mentioned eventNames.
Added a null check before mapping field "userIdentity.invokedBy".

2023-07-06

Added null check before mapping field "userIdentity.invokedBy".
Mapped "requestParameters.instanceType","requestParameters.instancesSet.items.0.minCount","requestParameters.instancesSet.items.0.maxCount" to "target.resource.attribute.labels".

2023-06-23

mapped logs to more specific "metadata.event_type" based on the field "eventname".
Mapped "target.resource.resource_type" as "VIRTUAL_MACHINE".
Mapped "requestParameters.status", "responseElements.certificate.status" to "target.resource.attribute.labels".
Mapped "requestParameters.instanceId" to "target.resource_ancestors.product_object_id".
Mapped "requestParameters.userName" to "target.user.userid".
Mapped "target.resource.name" and "target.resource.product_object_id" based upon keys present under each "eventName".
Mapped "userIdentity.arn" to "principal.resource.name".
Mapped "userIdentity.accountId" to "principal.resource.product_object_id".
For logs having "eventName" as following, mapped "metadata.event_type" to "RESOURCE_CREATION".
"CreateTrail","AllocateAddress","CreateVolume","CreateVirtualMFADevice","UploadSigningCertificate",
"CreateAccessKey","UploadSSHPublicKey","CreateServiceSpecificCredential","UploadCloudFrontPublicKey",
"CreateAnalyzer","CreateSAMLProvider","PutConfigurationRecorder","CreateRole","CreateInstanceProfile",
"CreateExportTask","CreateLogGroup","EnableSecurityHub","CreateEnvironment","CreateSession","CreateServiceLinkedRole",
"CreateSnapshot","CreateKeyPair","CreateSecurityGroup","CreateDetector","CreateFlowLogs",
"EnableMacie","ConnectDirectory","RunInstances","CreateImage","CreateOrganization"
For logs having "eventName" as following, mapped "metadata.event_type" to "RESOURCE_WRITTEN".
"StartLogging","StopLogging","AssociateAddress","DisassociateAddress","DetachVolume",
"AttachVolume","ModifyVolume","EnableMFADevice","ResyncMFADevice","UpdateSigningCertificate",
"UpdateAccessKey","UpdateSSHPublicKey","ResetServiceSpecificCredential","UpdateServiceSpecificCredential",
"UpdateCloudFrontPublicKey","DisableRegion","EnableRegion","UpdateSAMLProvider","StartConfigurationRecorder",
"StopConfigurationRecorder","PutRetentionPolicy","PutDataProtectionPolicy","UpdateDetector","UpdateMacieSession"
For logs having "eventName" as following, mapped "metadata.event_type" to "RESOURCE_DELETION".
"DeleteTrail","ReleaseAddress","DeleteVolume","DeactivateMFADevice","DeleteVirtualMFADevice",
"DeleteSigningCertificate","DeleteAccessKey","DeleteSSHPublicKey","DeleteServiceSpecificCredential",
"DeleteCloudFrontPublicKey","DeleteAnalyzer","DeleteSAMLProvider","DeleteConfigurationRecorder",
"DeletePolicy","DeleteRole","DeleteInstanceProfile","DeleteLogGroup","DisableSecurityHub","DisableMacie",
"DeleteSnapshot","DeleteDetector","DeleteFlowLogs","DeregisterImage","TerminateInstances"
For logs having "eventName" as following, mapped "metadata.event_type" to "RESOURCE_PERMISSIONS_CHANGE".
"AttachUserPolicy","DetachUserPolicy","PutUserPolicy","DeleteUserPolicy",
"PutUserPermissionsBoundary","DeleteUserPermissionsBoundary","AttachRolePolicy",
"DetachRolePolicy","PutRolePolicy","PutResourcePolicy","PutCredentials","DeleteDirectory"

2023-06-09

Modified the regex to identify the JSON Array logs.

2023-06-07

Mapped all the "principal.user" fields to "target.user" for "eventName" as "ConsoleLogin".

2023-05-26

Parsed logs of different josn pattern.
Mapped "cipherSuite" to "network.tls.cipher".
Mapped "requestID" to "target.resource.attribute.labels".
Mapped "assumedRoleId" to "security_result.about.resource.name".
Mapped "roleSessionName" to "target.resource.name".
Mapped "roleArn" to "target.resource.product_object_id".
Mapped "userAgent" to "network.http.user_agent".
Mapped "sourceIPAddress" to "principal.ip".
Mapped "sessionIssuer.userName" to "target.user.user_display_name".
Mapped "sessionIssuer.principalId" to "target.user.userid".
Mapped "userIdentity.accessKeyId" to "target.resource.product_object_id".
Mapped "userIdentity.arn" to "security_result.about.resource.id".
Mapped "req.detail.Longitude" to "_principal.location.region_longitude".
Mapped "req.detail.Latitude" to "_principal.location.region_latitude".
Mapped "detail.resourceType" to "target.resource.resource_subtype".
Set "security_result.alert_state" to "ALERTING".
Mapped "req.detail.recommendRemediation" to "security_result.action_details".
Mapped "eventLog.detail.eventName" to "metadata.product_event_type".

2023-02-23

Mapped "requestParameters.principalArn" to "principal.resource.name".
Mapped "resources.ARN" to "about.resource.name".

2022-11-24

Fix:
Parsed new format logs that has configurationItem by mapping following fields.
Mapped "configurationItem.awsAccountId" to "principal.user.userid".
Mapped "configurationItem.resourceId" to "target.resource.id".
Mapped "configurationItem.resourceType" to "target.resource.resource_subtype"
Mapped "configurationItem.awsRegion" to "target.location.country_or_region".
Mapped "configurationItem.configurationItemCaptureTime" to "target.asset.attribute.creation_time".
Mapped "configurationItem.configurationItemStatus" to "target.asset.attribute.labels".
Mapped "configurationItems.ARN" to "target.resource.attribute.labels".
Mapped "configurationItems.availabilityZone" to "target.resource.attribute.cloud.availability_zone".
Mapped "configurationItems.awsRegion" to "target.location.country_or_region".
Mapped "configurationItems.awsAccountId" to "principal.user.userid".
Mapped "configurationItems.configuration.activityStreamStatus" to "target.resource.attribute.labels".
Mapped "configurationItems.configuration.allocatedStorage" to "target.resource.attribute.labels".
Mapped "configurationItems.configuration.autoMinorVersionUpgrade" to "target.resource.attribute.labels".
Mapped "configurationItems.configuration.backupRetentionPeriod" to "target.resource.attribute.labels".
Mapped "configurationItems.configuration.copyTagsToSnapshot" to "target.resource.attribute.labels".
Mapped "configurationItems.configuration.dbClusterResourceId" to "target.resource.product_object_id".
Mapped "configurationItems.configuration.masterUsername" to "principal.user.user_display_name".
Mapped "configurationItems.resourceName" to "target.resource.name".

2022-10-13

For "eventName": "CreateAccessKey" mapped the field "responseElements.accessKey.accessKeyId" to "target.resource.product_object_id".
For "eventName": "UpdateAccessKey" mapped the field "requestParameters.accessKeyId" to "target.resource.product_object_id".
For "eventName": "DeleteAccessKey" mapped the field "requestParameters.accessKeyId" to "target.resource.product_object_id".
For "eventName": "CreateUser" mapped the field "responseElements.user.userId" to "target.user.product_object_id".
Mapped the field "eventTime" to "metadata.collected_timestamp".

2022-07-27

Added eventType "QueryDatabase" and mapped it"s fields.
Modified conditions for principal.ip or principal.host for handling new logs.
Changed the mapping of "requestParameters.roleArn", "requestParameters.registryId", "resources.accountId" from "target.resource.id" to "target.resource.product_object_id".
Modified the parsing condition for "req_params" to extract the values.

2022-07-08

Modified mapping for "req.requestParameters.roleName" from "target.user.role_name" to "target.user.attribute.roles".

2022-07-06

Changed mapping of "req.awsRegion" from "_principal.location.country_or_region" to "_principal.location.name".
Modified event_type from "GENERIC_EVENT" to "USER_LOGIN" for eventName "AssumeRole".
Modified event_type from "GENERIC_EVENT" to "USER_RESOURCE_ACCESS" for eventNAme "PutImage" or "GetDownloadUrlForLayer" or "BatchGetImage".
Modified event_type from "GENERIC_EVENT" to "USER_RESOURCE_DELETION" for eventName "DeleteNetworkInterface".

2022-06-06

For eventName "CreateUser/DeleteUser", modified condition for handling src mapping as existing one failed for new logs.
Modified puserId field to handle new unparsed log.

2022-05-27

Enhancement to map following raw logs elements to UDM elements:
"awsAccountId" mapped to "target.user.group_identifiers".
"digestS3Bucket" mapped to "target.resource.name".
"digestS3Object" mapped to "target.file.full_path".
"previousDigestHashValue" mapped to "target.file.sha256".
"digestSignatureAlgorithm" mapped to "event.idm.read_only_udm.additional.fields".
"digestPublicKeyFingerprint" mapped to "event.idm.read_only_udm.additional.fields".
"logFiles.s3Bucket" mapped to "about_resource.resource.name".
"logFiles.s3Object" mapped to "about_resource.file.full_path".
"logFiles.hashValue" mapped to "about_resource.file.sha256".

2022-05-27

Enhancement - Modified the value stored in metadata.product_name to "AWS CloudTrail".

2022-04-13

Enhancement to map following raw logs elements to UDM elements:
Mapped field "requestParameters.PublicAccessBlockConfiguration.IgnorePublicAcls", "requestParameters.CreateAccessPointRequest.PublicAccessBlockConfiguration.RestrictPublicBuckets", "requestParameters.CreateAccessPointRequest.PublicAccessBlockConfiguration.BlockPublicPolicy", "requestParameters.CreateAccessPointRequest.PublicAccessBlockConfiguration.BlockPublicAcls", "requestParameters.CreateAccessPointRequest.PublicAccessBlockConfiguration.IgnorePublicAcls", "additionalEventData.configRuleInputParameters.RestrictPublicBuckets", "additionalEventData.configRuleInputParameters.BlockPublicPolicy", "additionalEventData.configRuleInputParameters.BlockPublicAcls", "additionalEventData.configRuleInputParameters.IgnorePublicAcls" to "target.resource.attribute.labels".