Collect Carbon Black EDR logs

Supported in:

This document explains how to collect Carbon Black EDR logs from Cloud and On-Prem environments using AWS S3. The parser extracts fields from JSON, CSV, or syslog formatted messages, normalizes them, and maps them to the UDM. It handles various Carbon Black event types, including network connections, process events, file modifications, registry changes, and IOC hits, enriching the data with threat intelligence and device information where available.

Before you begin

Ensure that you have:

  • A configured Google SecOps instance.
  • Privileged access to AWS IAM and S3.
  • Privileged access to Cloud or On-Prem Carbon Black EDR.

Configure Carbon Black EDR On-Prem

Configure Amazon S3 bucket for On-Prem

  1. Create an Amazon S3 bucket following this user guide: Creating a bucket
  2. Save the bucket Name and Region for later use.
  3. Create a user following this user guide: Creating an IAM user.
  4. Select the created User.
  5. Select the Security credentials tab.
  6. Click Create Access Key in the Access Keys section.
  7. Select Third-party service as the Use case.
  8. Click Next.
  9. Optional: add a description tag.
  10. Click Create access key.
  11. Click Download CSV file to save the Access Key and Secret Access Key for later use.
  12. Click Done.
  13. Select the Permissions tab.
  14. Click Add permissions in the Permissions policies section.
  15. Select Add permissions.
  16. Select Attach policies directly.
  17. Search for and select the AmazonS3FullAccess policy.
  18. Click Next.
  19. Click Add permissions.

Install cb-event-forwarder on On-Prem EDR Server

  1. Install the CbOpenSource repository if it isn't already present:

    cd /etc/yum.repos.d
curl -O https://opensource.carbonblack.com/release/x86_64/CbOpenSource.repo

  2. Install the RPM using YUM:

    yum install cb-event-forwarder

  3. If you're using EDR 7.1.0 or greater, run the following script to set the appropriate permissions needed by EDR:

    /usr/share/cb/integrations/event-forwarder/cb-edr-fix-permissions.sh

Configure cb-event-forwarder to Output JSON Logs

  1. Open the configuration file:

    sudo nano /etc/cb/integrations/event-forwarder/cb-event-forwarder.conf

  2. Modify the following parameters:

    [event_forwarder]
output_format=json   # Enable JSON format
output_type=s3       # Send logs to AWS S3
s3_bucket_name=YOUR-S3-BUCKET-NAME
s3_region=YOUR-S3-BUCKET-NAME
s3_access_key_id=YOUR_AWS_ACCESS_KEY
s3_secret_access_key=YOUR_AWS_SECRET_KEY
s3_prefix=carbonblack/edr/logs

  3. Save and exit using the keyboard:

    • Ctrl + X, then Y and Enter.

  4. Start cb-event-forwarder:

    sudo systemctl enable cb-event-forwarder
sudo systemctl restart cb-event-forwarder
sudo systemctl status cb-event-forwarder

Configure Carbon Black Cloud Event Forwarder for S3

Create an AWS S3 Bucket

  1. Sign in to the AWS Management Console.
  2. Ensure that the AWS region matches the region of the Event Forwarder:
    1. In the AWS Console page, locate the region.
    2. Use the drop-down to select the correct region of your Event Forwarder.
    3. The following list gives the applicable AWS Region for each Carbon Black EDR URL.
      • "instance-alias".my.carbonblack.io - Region: US East (N. Virginia) (us-east-1)
      • "instance-alias".my.cbcloud.de - Region: Europe (Frankfurt) (eu-central-1)
      • "instance-alias".my.cbcloud.sg Region: Asia Pacific (Singapore) (ap-southeast-1)
  3. Select Services.
  4. Go to the S3 console.
  5. Click Create bucket to open the Create bucket wizard.
    1. In Bucket name, enter a unique name for your bucket (for example, CB-EDR).
    2. Ensure the Region defaults to the one you selected earlier.
    3. Update the Block Public Access defaults to allow public access (this is required for ingesting the logs into Google SecOps).
    4. Select Create Bucket.

Configure S3 Bucket to allow the Event Forwarder to write events

  1. Create a User following this user guide: Creating an IAM user.
  2. Select the created User.
  3. Select the Security credentials tab.
  4. Click Create Access Key in the Access Keys section.
  5. Select Third-party service as the Use case.
  6. Click Next.
  7. Optional: add a description tag.
  8. Click Create access key.
  9. Click Download CSV file to save the Access Key and Secret Access Key for later use.
  10. Click Done.
  11. Select the Permissions tab.
  12. Click Add permissions in the Permissions policies section.
  13. Select Add permissions.
  14. Select Attach policies directly.
  15. Search for the AmazonS3FullAccess policy.
  16. Select the policy.
  17. Click Next.
  18. Click Add permissions.

Configure Events forwarding in the EDR Console

  1. Sign in to VMware Carbon Black Cloud.
  2. Go to the event forwarder tab
  3. Enable the events you would like the product to upload to S3.
  4. Go to Output and Type and set to S3.
  5. Provide the S3 bucket name in the following format <region>:<bucket-name> (for example, us-east-1:cb-edr).
  6. Select upload AWS credentials file in INI format.

  7. The following is an example of a profile:

    AWS Access Key ID [None]: AKIAIOSFODNN7EXAMPLE
AWS Secret Access Key [None]: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
Default region name [None]: us-east-1

  8. Click Save and restart the service for the changes to take effect.

Configure a feed in Google SecOps to ingest the Carbon Black EDR logs

  1. Go to SIEM Settings > Feeds.
  2. Click Add new.
  3. In the Feed name field, enter a name for the feed (for example, Carbon Black EDR Logs).
  4. Select Amazon S3 as the Source type.
  5. Select Carbon Black EDR as the Log type.
  6. Click Next.

  7. Specify values for the following input parameters:

    • Region: the region where the Amazon S3 bucket is located.
    • S3 URI: the bucket URI.
      • s3:/BUCKET_NAME
        • Replace BUCKET_NAME with the actual name of the bucket.
    • URI is a: select the URI_TYPE according to the log stream configuration (Single file | Directory | Directory which includes subdirectories).
    • Source deletion options: select the deletion option according to your preference.
    • Access Key ID: the User access key with access to the S3 bucket.
    • Secret Access Key: the User secret key with access to the S3 bucket.
    • Asset namespace: the asset namespace.
    • Ingestion labels: the label to be applied to the events from this feed.

  8. Click Next.

  9. Review your new feed configuration in the Finalize screen, and then click Submit.

UDM Mapping Table

Log Field UDM Mapping Logic
action security_result.detection_fields[?key == 'action'].value The value of the action field from the raw log.
cb_server intermediary.hostname The value of the cb_server field from the raw log.
cb_version metadata.product_version The value of the cb_version field from the raw log.
child_pid target.process.pid (for ingress.event.childproc events) The value of the child_pid field from the raw log when the type is ingress.event.childproc.
child_process_guid target.process.product_specific_process_id (for ingress.event.childproc events) "CB:" concatenated with the value of the child_process_guid field from the raw log when the type is ingress.event.childproc.
child_username target.user.userid (for ingress.event.childproc events) The value of the child_username field from the raw log when the type is ingress.event.childproc.
childproc_guid target.process.product_specific_process_id (for endpoint.event.procstart events) "CB:" concatenated with the value of the childproc_guid field from the raw log when the type is endpoint.event.procstart.
childproc_hash.0 target.process.file.md5 (for endpoint.event.procstart events) The first element of the childproc_hash array from the raw log when the type is endpoint.event.procstart.
childproc_hash.1 target.process.file.sha256 (for endpoint.event.procstart events) The second element of the childproc_hash array from the raw log when the type is endpoint.event.procstart.
childproc_name target.process.file.full_path (for endpoint.event.procstart events) The value of the childproc_name field from the raw log when the type is endpoint.event.procstart.
childproc_pid target.process.pid (for endpoint.event.procstart events) The value of the childproc_pid field from the raw log when the type is endpoint.event.procstart.
childproc_publisher.0.name security_result.detection_fields[?key == 'childproc_publisher_name'].value (for endpoint.event.procstart events) "childproc_publisher_name:" concatenated with the value of childproc_publisher.0.name from the raw log when the type is endpoint.event.procstart.
childproc_publisher.0.state security_result.detection_fields[?key == 'childproc_publisher_state'].value (for endpoint.event.procstart events) "childproc_publisher_state:" concatenated with the value of childproc_publisher.0.state from the raw log when the type is endpoint.event.procstart.
childproc_reputation security_result.detection_fields[?key == 'childproc_reputation'].value (for endpoint.event.procstart events) The value of the childproc_reputation field from the raw log when the type is endpoint.event.procstart.
childproc_username target.user.userid (for endpoint.event.procstart events) The value of the childproc_username field from the raw log when the type is endpoint.event.procstart.
clientIp principal.ip, principal.asset.ip The value of the clientIp field from the raw log.
cmdline target.process.command_line (for feed.query.hit.process and feed.storage.hit.process events), additional.fields[?key == 'cmdline_*'].value.string_value (for watchlist.storage.hit.process events) The value of the cmdline field from the raw log when the type is feed.query.hit.process or feed.storage.hit.process. For watchlist.storage.hit.process events, it's stored in additional.fields with the key "cmdline_*".
command_line target.process.command_line (for ingress.event.procstart events) The value of the command_line field from the raw log when the type is ingress.event.procstart.
comms_ip intermediary.ip The value of the comms_ip field from the raw log.
computer_name principal.hostname, principal.asset.hostname The value of the computer_name field from the raw log.
crossproc_api additional.fields[?key == 'crossproc_api'].value.string_value (for endpoint.event.apicall events) The value of the crossproc_api field from the raw log when the type is endpoint.event.apicall.
crossproc_guid additional.fields[?key == 'crossproc_guid'].value.string_value (for endpoint.event.crossproc events) The value of the crossproc_guid field from the raw log when the type is endpoint.event.crossproc.
crossproc_hash.0 additional.fields[?key == 'crossproc_md5'].value.string_value (for endpoint.event.crossproc events) The first element of the crossproc_hash array from the raw log when the type is endpoint.event.crossproc.
crossproc_hash.1 additional.fields[?key == 'crossproc_sha256'].value.string_value (for endpoint.event.crossproc events) The second element of the crossproc_hash array from the raw log when the type is endpoint.event.crossproc.
crossproc_name target.process.file.full_path (for endpoint.event.crossproc events) The value of the crossproc_name field from the raw log when the type is endpoint.event.crossproc.
crossproc_publisher.0.name security_result.detection_fields[?key == 'crossproc_publisher_name'].value (for endpoint.event.crossproc events) "crossproc_publisher_name:" concatenated with the value of crossproc_publisher.0.name from the raw log when the type is endpoint.event.crossproc.
crossproc_publisher.0.state security_result.detection_fields[?key == 'crossproc_publisher_state'].value (for endpoint.event.crossproc events) "crossproc_publisher_state:" concatenated with the value of crossproc_publisher.0.state from the raw log when the type is endpoint.event.crossproc.
crossproc_reputation additional.fields[?key == 'crossproc_reputation'].value.string_value (for endpoint.event.crossproc events) The value of the crossproc_reputation field from the raw log when the type is endpoint.event.crossproc.
crossproc_target additional.fields[?key == 'crossproc_target'].value.string_value (for endpoint.event.crossproc events) The value of the crossproc_target field from the raw log when the type is endpoint.event.crossproc. Converted to a string "true" or "false".

Changes

2024-05-13

  • Mapped "alert_url" field to "metadata.url_back_to_product" UDM field.

2024-01-19

  • Added a null check for "filemod_hash.0" and "filemod_hash.1" before mapping.

2023-12-27

  • Initialized "filemod_hash.0" and "filemod_hash.1" to null to parse the unparsed logs.

2023-10-26

  • Added "gsub" function to parse the unparsed fields.

2023-10-13

  • Handled new JSON logs by adding JSON block.
  • Removed redundant code for fields "computer_name", "parent_name", "process_name", "pid", "process_path", "md5", "sha256", "process_guid", "parent_pid", "docs.0.process_pid", "cb_version", "process_hash.0", "process_hash.1", "parent_hash.0" and "parent_hash.1".

2023-07-21

  • Added MITRE ATT&CK tactic and technique details to "security_result.attack_details".

2023-03-24

  • Mapped the field "protocol" to "network.ip_protocol".
  • Added null conditional check for the field "child_username", "child_pid", "child_command_line".
  • Changed the "metadata.event_type" from "GENERIC_EVENT" to "STATUS_UPDATE" when "principal.hostname" or "principal.ip" is not null.

2023-03-14

  • Bug-fix:
  • Mapped the following fields when the field "type" is null:
  • Mapped the field "process_guid" to "principal.process.product_specific_process_id".
  • Mapped the field "device_external_ip" to "target.ip".
  • Mapped the field "device_os" to "principal.platform".
  • Mapped the field "device_group" to "principal.group.group_display_name".
  • Mapped the field "process_pid" to "principal.process.pid".
  • Mapped the field "process_path" to "principal.process.file.full_path".
  • Mapped the field "process_cmdline" to "principal.process.command_line".
  • Mapped the field "process_hash.0" to "principal.process.file.md5".
  • Mapped the field "principal.1" to "principal.process.file.sha256".
  • Mapped the field "process_username" to "principal.user.userid".
  • Mapped the field "clientIp" to "principal.ip".
  • Mapped the field "description" to "metadata.description".
  • Mapped the field "orgName" to "principal.administrative_domain".
  • Mapped the following fields when the field "ruleName" contains "CYDERES":
  • Mapped the field "deviceInfo.internalIpAddress" to "principal.ip".
  • Mapped the field "deviceInfo.externalIpAddress" to "target.ip".
  • Mapped the field "ruleName" to "security_result.rule_name".
  • Mapped the field "deviceInfo.deviceType" to "principal.asset.platform_software.platform".
  • Mapped the field "domain" to "principal.administrative_domain".
  • Mapped the field "deviceInfo.groupName" to "principal.group.group_display_name".
  • Mapped the field "deviceInfo.deviceVersion" to "principal.asset.platform_software.platform_version".
  • Mapped the field "deviceInfo.deviceId" to "principal.asset.asset_id".
  • Mapped the field "eventId" to "additional.fields".
  • Changed the "metadata.event_type" from "GENERIC_EVENT" to "NETWORK_CONNECTION" when "principal.ip" and "target.ip" is not null.
  • Changed the "metadata.event_type" from "GENERIC_EVENT" to "STATUS_UPDATE" when "principal.ip" is not null.

2023-02-03

  • Bug-fix: Map "filemod_hash" to "target.file" instead of "target.process.file".

2023-01-20

  • Bug-fix: Stopped populating and mapping product_specific_process_id for empty process ids.

2022-11-25

  • Mapped 'remote_ip' to 'principal.ip' and 'local_ip' to 'target.ip' for 'Inbound' TCP/UDP events.
  • Mapped 'remote_port' to 'principal.port' and 'local_port' to 'target.port' for 'Inbound' TCP/UDP events.

2022-10-06

  • Migrated all customer specific parsers to default parser.

2022-07-10

  • Updated mapping of 'event_type' to 'PROCESS_LAUNCH' for logs of type 'endpoint.event.'.

Need more help? Get answers from Community members and Google SecOps professionals.