Stay organized with collections Save and categorize content based on your preferences.

Ingest AWS Logs into Chronicle

Overview

This document details the steps to configure AWS Cloudtrail logs for ingestion into Chronicle. These steps are also applicable for ingesting logs from other AWS services e.g. AWS GuardDuty, AWS VPC Flow, AWS CloudWatch, AWS Security Hub, and others into Chronicle.

The first part of this document details the concise steps. The second part outlines the same steps with screenshots.

Configure AWS Cloudtrail (or other service)

In this step, we will configure AWS Cloudtrail logs and direct them to the written in the AWS S3 bucket that we created in the previous step.

  1. In the AWS console, search for Cloudtrail.
  2. Click Create trail.
  3. Provide a Trail name.
  4. Select Create new S3 bucket. You may also choose to use an existing S3 bucket.
  5. Provide a name for AWS KMS alias, or choose an existing AWS KMS Key.
  6. You can leave the other settings as default, and click Next.
  7. Choose Event type, add Data events as required, and click Next.
  8. Review the settings in Review and create and click Create trail.
  9. In the AWS console, search for Amazon S3 Buckets.
  10. Click the newly created log bucket, and select the folder AWSLogs. Then click Copy S3 URI and save it for use in the following steps.

Configure AWS IAM User

In this step, we will configure an AWS IAM user which Chronicle will use to get log feeds from AWS.

  1. In the AWS console, search for IAM.
  2. Click Users, and then in the following screen, click Add Users.
  3. Provide a name for the user, e.g. chronicle-feed-user, Select AWS credential type as Access key - Programmatic access and click Next: Permissions.
  4. In the next step, select Attach existing policies directly and select AmazonS3ReadOnlyAccess or AmazonS3FullAccess, as required. AmazonS3FullAccess would be used if Chronicle should clear the S3 buckets after reading logs, to optimize AWS S3 storage costs.
  5. As a recommended alternative to the previous step, you can further restrict access to only the specified S3 bucket by creating a custom policy. Click Create policy and follow the AWS documentation to create a custom policy.
  6. Click Next:Tags.
  7. Add any tags if required, and click Next:Review.
  8. Review the configuration and click Create user.
  9. Copy the Access key ID and Secret access key of the created user, for use in the next step.

Configure Feed in Chronicle to Ingest AWS Logs

  1. Go to Chronicle settings, and click Feeds.
  2. Click Add New.
  3. Select Amazon S3 for Source Type.
  4. Select AWS Cloudtrail (or other AWS service) for Log Type.
  5. Click Next.
  6. Select region and provide S3 URI of the Amazon S3 bucket you copied earlier. Further, you could append the S3 URI with:

    
    {{datetime("yyyy/MM/dd")}}
    
    

    As in the following example, so that Chronicle would scan logs each time only for a particular day:

    
    s3://aws-cloudtrail-logs-XXX-1234567/AWSLogs/1234567890/CloudTrail/us-east-1/{{datetime("yyyy/MM/dd")}}/
    
    
  7. Under URI IS A select Directories including subdirectories. Select an appropriate option under Source Deletion Option, this should match with the permissions of the IAM User account you created earlier.

  8. Provide Access Key ID and Secret Access Key of the IAM User account you created earlier.

  9. Click Next and Finish.

Following is the same document as above with screenshots:

Configure AWS Cloudtrail (or other service)

Complete the following steps to configure AWS Cloudtrail logs and direct these logs to be written to the AWS S3 bucket created in the previous procedure:

  1. In the AWS console, search for Cloudtrail.
  2. Click Create trail.

    alt_text

  3. Provide a Trail name.

  4. Select Create new S3 bucket. You may also choose to use an existing S3 bucket.

  5. Provide a name for AWS KMS alias, or choose an existing AWS KMS Key.

    alt_text

  6. You can leave the other settings as default, and click Next.

  7. Choose** Event type**, add Data events as required, and click Next.

    alt_text

  8. Review the settings in Review and create and click Create trail.

  9. In the AWS console, search for Amazon S3 Buckets.

    alt_text

  10. Click the newly created log bucket, and select the folder AWSLogs. Then click Copy S3 URI and save it for use in the following steps.

    alt_text

Configure AWS IAM User

In this step, we will configure an AWS IAM user which Chronicle will use to get log feeds from AWS.

  1. In the AWS console, search for IAM.

    alt_text

  2. Click Users, and then in the following screen, click Add Users.

    alt_text

  3. Provide a name for the user, e.g. chronicle-feed-user, Select AWS credential type as Access key - Programmatic access and click Next: Permissions.

    alt_text

  4. In the next step, select Attach existing policies directly and select AmazonS3ReadOnlyAccess or AmazonS3FullAccess, as required. AmazonS3FullAccess would be used if Chronicle should clear the S3 buckets after reading logs, to optimize AWS S3 storage costs. Click Next:Tags.

    alt_text

  5. As a recommended alternative to the previous step, you can further restrict access to only the specified S3 bucket by creating a custom policy. Click Create policy and follow the AWS documentation to create a custom policy.

    alt_text

  6. Add any tags if required, and click Next:Review.

  7. Review the configuration and click Create user.

    alt_text

  8. Copy the Access key ID and Secret access key of the created user, for use in the next step.

    alt_text

Configure Feed in Chronicle to Ingest AWS Logs

  1. Go to Chronicle settings, and click Feeds.
  2. Click Add New.
  3. Select Amazon S3 for Source Type.
  4. Select AWS Cloudtrail (or other AWS service) for Log Type.

alt_text

  1. Click Next.
  2. Select region and provide S3 URI of the Amazon S3 bucket you copied earlier. Further you could append the S3 URI with:

    
    {{datetime("yyyy/MM/dd")}}
    
    

    As in the following example, so that Chronicle would scan logs each time only for a particular day:

    
    s3://aws-cloudtrail-logs-XXX-1234567/AWSLogs/1234567890/CloudTrail/us-east-1/{{datetime("yyyy/MM/dd")}}/
    
    
  3. Under URI IS A select Directories including subdirectories. Select an appropriate option under Source Deletion Option, this should match with the permissions of the IAM User account we created earlier.

  4. Provide Access Key ID and Secret Access Key of the IAM User account we created earlier. alt_text

  5. Click Next and Finish.