Collect AWS Macie logs

Supported in:

Google SecOps SIEM

Note: This feature is covered by Pre-GA Offerings Terms of the Google Security Operations Service Specific Terms. Pre-GA features might have limited support, and changes to pre-GA features might not be compatible with other pre-GA versions. For more information, see the Google Security Operations Technical Support Service guidelines and the Google Security Operations Service Specific Terms.

This document explains how to ingest AWS Macie logs to Google Security Operations. AWS Macie is a security service that uses machine learning to automatically discover, classify, and protect sensitive data. This integration will allow you to send Macie logs to Google SecOps for enhanced analysis and monitoring.

Before You Begin

• Ensure that you have a Google SecOps instance.
• Ensure that you have privileged access to AWS.

Configure Amazon S3 and IAM

1. Create an Amazon S3 bucket following this user guide: Creating a bucket
2. Save the bucket Name and Region for later use.
3. Create a user following this user guide: Creating an IAM user.
4. Select the created User.
5. Select the Security credentials tab.
6. Click Create Access Key in the Access Keys section.
7. Select Third-party service as the Use case.
8. Click Next.
9. Optional: add a description tag.
10. Click Create access key.
11. Click Download CSV file to save the Access Key and Secret Access Key for later use.
12. Click Done.
13. Select the Permissions tab.
14. Click Add permissions in the Permissions policies section.
15. Select Add permissions.
16. Select Attach policies directly.
17. Search for and select the AmazonS3FullAccess policy.
18. Click Next.
19. Click Add permissions.

Optional: Configuring AWS Macie

1. Sign in to the AWS Management Console.
2. In the search bar, type and select Macie from the services list.
3. Click Create job.
4. Create a new bucket or proceed with the existing one.
5. Add Schedule job.
6. Select all Managed Data Identifiers.
7. Skip Select Custom Data Identifiers and click Next.
8. Skip Select Allow list and click Next.
9. Provide a meaningful name and description.
10. Click Next.
11. Review and click Submit.

Configure CloudTrail for AWS Macie

1. Sign in to the AWS Management Console.

2. In the search bar, type and select CloudTrail from the services list.

   Note: If you already have a trail, you can modify and use it.

3. If you want to proceed with a new trail, click Create trail.

4. Provide a Trail name (for example, Macie-Activity-Trail).

5. Select the Enable for all accounts in my organization checkbox.

6. Type the S3 bucket URI created earlier (the format should be: s3://your-log-bucket-name/), or create a new S3 bucket.

7. If SSE-KMS is enabled, provide a name for AWS KMS alias, or choose an existing AWS KMS Key.

8. You can leave the other settings as default.

9. Click Next.

10. Select Management events and Data events under Event Types.

11. Click Next.

12. Review the settings in Review and create.

13. Click Create trail.

14. Optional: if you created a new bucket, continue with the following process:

    1. Go to S3.
    2. Identify and select the newly created log bucket.
    3. Select the folder AWSLogs.
    4. Click Copy S3 URI and save it.

Configure a feed in Google SecOps to ingest AWS Macie logs

1. Go to SIEM Settings > Feeds.
2. Click Add new.
3. In the Feed name field, enter a name for the feed (for example, AWS Macie Logs).
4. Select Amazon S3 as the Source type.
5. Select AWS Macie as the Log type.
6. Click Next.

7. Specify values for the following input parameters:

   • Region: the region where the Amazon S3 bucket is located.
   • S3 URI: the bucket URI.
     • s3://your-log-bucket-name/
       • Replace your-log-bucket-name with the actual name of the bucket.
   • URI is a: select Directory or Directory which includes subdirectories.

   • Source deletion options: select the deletion option according to your preference.

     Note: If you select the Delete transferred files or Delete transferred files and empty directories option, make sure that you granted appropriate permissions to the service account.

   • Access Key ID: the User access key with access to the S3 bucket.

   • Secret Access Key: the User secret key with access to the S3 bucket.

   • Asset namespace: the asset namespace.

   • Ingestion labels: the label to be applied to the events from this feed.

8. Click Next.

9. Review your new feed configuration in the Finalize screen, and then click Submit.

UDM Mapping Table

Log Field	UDM Mapping	Logic
accountId	principal.group.product_object_id	Directly mapped from the accountId field.
category	security_result.category_details	Directly mapped from the category field.
classificationDetails.jobArn	security_result.rule_name	Directly mapped from the classificationDetails.jobArn field.
classificationDetails.jobId	security_result.rule_id	Directly mapped from the classificationDetails.jobId field.
classificationDetails.originType	security_result.rule_type	Directly mapped from the classificationDetails.originType field.
classificationDetails.result.mimeType	target.file.mime_type	Directly mapped from the classificationDetails.result.mimeType field.
classificationDetails.result.sensitiveData.category	security_result.detection_fields.value	Directly mapped from the classificationDetails.result.sensitiveData.category field. The parser iterates through the sensitiveData array and creates multiple detection_fields objects.
classificationDetails.result.sensitiveData.totalCount	security_result.detection_fields.value	Directly mapped from the classificationDetails.result.sensitiveData.totalCount field. The parser iterates through the sensitiveData array and creates multiple detection_fields objects.
createdAt	metadata.event_timestamp	Parsed and converted to UDM timestamp format from the createdAt field.
description	security_result.description	Directly mapped from the description field.
id	metadata.product_log_id	Directly mapped from the id field. Hardcoded to SCAN_FILE in the parser. Taken from the top-level log_type field in the raw log. Hardcoded to AWS Macie in the parser. Directly mapped from the schemaVersion field. Hardcoded to AMAZON in the parser. Concatenated from resourcesAffected.s3Bucket.name, region, and the string ".s3.amazonaws.com".
region	target.location.name	Directly mapped from the region field.
resourcesAffected.s3Bucket.arn	target.resource_ancestors.product_object_id	Directly mapped from the resourcesAffected.s3Bucket.arn field.
resourcesAffected.s3Bucket.createdAt	target.resource_ancestors.attribute.creation_time	Parsed and converted to UDM timestamp format from the resourcesAffected.s3Bucket.createdAt field.
resourcesAffected.s3Bucket.name	target.resource_ancestors.name	Directly mapped from the resourcesAffected.s3Bucket.name field.
resourcesAffected.s3Bucket.owner.displayName	target.user.user_display_name	Directly mapped from the resourcesAffected.s3Bucket.owner.displayName field.
resourcesAffected.s3Bucket.owner.id	target.user.userid	Directly mapped from the resourcesAffected.s3Bucket.owner.id field.
resourcesAffected.s3Object.eTag	target.file.md5	Directly mapped from the resourcesAffected.s3Object.eTag field.
resourcesAffected.s3Object.key	target.file.names	Directly mapped from the resourcesAffected.s3Object.key field.
resourcesAffected.s3Object.key	target.resource.name	Directly mapped from the resourcesAffected.s3Object.key field.
resourcesAffected.s3Object.lastModified	target.resource.attribute.last_update_time	Parsed and converted to UDM timestamp format from the resourcesAffected.s3Object.lastModified field.
resourcesAffected.s3Object.path	target.file.full_path	Prefixed with "s3://" and mapped from the resourcesAffected.s3Object.path field.
resourcesAffected.s3Object.path	target.resource.product_object_id	Directly mapped from the resourcesAffected.s3Object.path field.
resourcesAffected.s3Object.size	target.file.size	Directly mapped from the resourcesAffected.s3Object.size field after converting to unsigned integer.
resourcesAffected.s3Object.storageClass	target.resource.attribute.labels.value	Directly mapped from the resourcesAffected.s3Object.storageClass field. The key is hardcoded to "storageClass". Hardcoded to DATA_AT_REST in the parser.
security_result.detection_fields.key	category, totalCount	Hardcoded keys for the detection fields.
severity.description	security_result.severity	Mapped from the severity.description field. "Low" is mapped to LOW, "Medium" to MEDIUM, and "High" to HIGH. Hardcoded to AMAZON_WEB_SERVICES in the parser. Hardcoded to STORAGE_OBJECT in the parser. Hardcoded to STORAGE_BUCKET in the parser.
title	security_result.summary	Directly mapped from the title field.
type	metadata.product_event_type	Directly mapped from the type field.

Changes

2022-08-08

• Newly created parser.

Need more help? Get answers from Community members and Google SecOps professionals.