Collect AWS Config logs

Supported in:

Google SecOps SIEM

Note: This feature is covered by Pre-GA Offerings Terms of the Google Security Operations Service Specific Terms. Pre-GA features might have limited support, and changes to pre-GA features might not be compatible with other pre-GA versions. For more information, see the Google Security Operations Technical Support Service guidelines and the Google Security Operations Service Specific Terms.

This document explains how to create a new S3 bucket to store the CloudTrail logs and how to create an IAM user to retrieve the log feeds from AWS. AWS Config provides a detailed view of the configuration of AWS resources in your AWS account. This includes how the resources are related to one another and how they were configured in the past so that you can see how the configurations and relationships change over time.

Before you begin

• Ensure that you have a Google SecOps instance.
• Ensure that you have privileged access to AWS.

Configure CloudTrail and AWS S3 bucket

1. Sign in to the AWS Management Console.
2. Go to the Amazon S3 console.
3. In the AWS console, search for Cloudtrail.
4. Click Create trail.
5. Provide a Trail name.
6. Select Create new S3 bucket (you may also choose to use an existing S3 bucket).

7. Provide a name for the AWS KMS alias, or choose an existing AWS KMS Key.

   Note: Leave the other settings as default.

8. Click Next.

9. Choose Event type and add Data events.

10. Click Next.

11. Review the settings and click Create trail.

12. In the AWS console, search for S3 Buckets.

13. Click the newly created log bucket, and select the AWSLogs folder.

14. Click Copy S3 URI and save it.

Configure AWS Config API Calls Logging

1. In AWS, go to AWS Config > Set up AWS Config.
2. Select the bucket type (either select the existing bucket details or create a new one).
3. Select all required AWS-managed rules and click Next to select a bucket.
4. Refer to AWS Config for details on rule types to help you select the appropriate rule based on your requirements:
   • Compliance rules: allow to evaluate the configurations of resources to ensure that they meet compliance standards or regulatory requirements.
   • Configuration rules: allow to evaluate the configurations of resources to ensure that they meet the required configuration standards.
   • Performance rules: allow to evaluate the configurations of resources to ensure that they are optimized for performance.
   • Security rules: allow to evaluate the configurations of resources to ensure that they meet security standards or requirements.
5. Click Create config.
6. Go to Amazon S3.
7. Click the newly created log bucket, and select the folder AWSLogs.
8. Click Copy S3 URI and save it.

Configure AWS IAM User

1. In the AWS console, search for IAM.
2. Click Users.
3. Click Add Users.
4. Provide a name for the user (for example, chronicle-feed-user).
5. Select Access key - Programmatic access as the AWS credential type.
6. Click Next: Permissions.
7. Select Attach existing policies directly.
8. Select AmazonS3ReadOnlyAccess or AmazonS3FullAccess.

Note: Use AmazonS3FullAccess if Google SecOps needs to clear the S3 buckets after reading logs to optimize AWS S3 storage costs.

1. Click Next: Tags.
2. Optional: Add any tags if required.
3. Click Next: Review.
4. Review the configuration and click Create user.
5. Copy the Access key ID and Secret access key of the created user.

Configure a feed in Google SecOps to ingest the AWS Config logs

1. Go to SIEM Settings > Feeds.
2. Click Add new.
3. In the Feed name field, enter a name for the feed (for example, AWS Config Logs).
4. Select Amazon S3 as the Source type.
5. Select AWS Config as the Log type.
6. Click Next.

7. Specify values for the following input parameters:

   • Region: the region where the Amazon S3 bucket is located.
   • S3 URI: the bucket URI.
     • s3:/BUCKET_NAME
       • Replace BUCKET_NAME with the actual name of the bucket.
   • URI is a: select the URI_TYPE according to the log stream configuration (Single file | Directory | Directory which includes subdirectories).
   • Source deletion options: select the deletion option according to your preference.
   Note: If you select the Delete transferred files or Delete transferred files and empty directories option, make sure that you granted appropriate permissions to the service account.
   • Access Key ID: the User access key with access to the S3 bucket.
   • Secret Access Key: the User secret key with access to the S3 bucket.
   • Asset namespace: the asset namespace.
   • Ingestion labels: the label to be applied to the events from this feed.

8. Click Next.

9. Review your new feed configuration in the Finalize screen, and then click Submit.

UDM Mapping

Log field	UDM mapping	Logic
ARN	target.resource.id	The value is taken from the ARN field.
awsAccountId	principal.user.userid	The value is taken from the awsAccountId field.
awsRegion	target.asset.location.country_or_region	The value is taken from the awsRegion field.
configurationItem.awsAccountId	principal.user.userid	The value is taken from the configurationItem.awsAccountId field.
configurationItem.configurationItemCaptureTime	target.asset.attribute.creation_time	The value is taken from the configurationItem.configurationItemCaptureTime field and converted to a timestamp.
configurationItem.configurationItemStatus	target.asset.attribute.labels.value	The value is taken from the configurationItem.configurationItemStatus field. The key is set to "Configuration Item Status".
configurationItem.relationships.name	additional.fields.value.list_value.values.string_value	The value is taken from the configurationItem.relationships.name field. The key is set to "configurationItem.relationships.resource_names".
configurationItem.relationships.resourceId	additional.fields.value.list_value.values.string_value	The value is taken from the configurationItem.relationships.resourceId field. The key is set to "configurationItem.relationships.resource_ids".
configurationItem.relationships.resourceType	additional.fields.value.list_value.values.string_value	The value is taken from the configurationItem.relationships.resourceType field. The key is set to "configurationItem.relationships.resource_types".
configurationItem.resourceId	target.resource.id	The value is taken from the configurationItem.resourceId field.
configurationItem.resourceType	target.resource.resource_subtype	The value is taken from the configurationItem.resourceType field.
N/A	metadata.event_type	If configurationItemDiff.changeType is "UPDATE", metadata.event_type is set to "RESOURCE_WRITTEN". If configurationItemDiff.changeType is "CREATE", metadata.event_type is set to "RESOURCE_CREATION". If configurationItem.configurationItemStatus is "OK" or "ResourceDiscovered", metadata.event_type is set to "RESOURCE_READ". If configurationItem.configurationItemStatus is "ResourceDeleted", metadata.event_type is set to "RESOURCE_DELETION". If none of these conditions are met, metadata.event_type is set to "GENERIC_EVENT".
N/A	metadata.log_type	Set to "AWS_CONFIG".
N/A	metadata.product_name	Set to "AWS Config".
N/A	metadata.vendor_name	Set to "AMAZON".
N/A	target.asset.attribute.cloud.environment	Set to "AMAZON_WEB_SERVICES".
N/A	target.resource.resource_type	Set to "VIRTUAL_MACHINE".

Changes

2024-02-22

• Moved data previously stored in specific fields related to "configurationItem.relationships" to a more general field called "additional.fields".

2022-05-27

• The parser now explicitly labels its output as coming from "AWS Config".

2022-03-30

• Improved how the parser handles "relationship.resourceId" information, making it work across more log types.

Need more help? Get answers from Community members and Google SecOps professionals.