Stay organized with collections Save and categorize content based on your preferences.

Feed Management API

How to authenticate with the Chronicle API

This Chronicle API uses the OAuth 2.0 protocol for authentication and authorization. Your application can complete these tasks using either of the following implementations:

  • Using the Google API Client Library for your computer language.

  • Directly interfacing with the OAuth 2.0 system using HTTP.

See the reference documentation for the Google Authentication library in Python.

Google Authentication libraries are a subset of the Google API client libraries. See other language implementations.

Getting API authentication credentials

Your Chronicle representative will provide you with a Google Developer Service Account Credential to enable the API client to communicate with the API.

You also must provide the Auth Scope when initializing your API client. OAuth 2.0 uses a scope to limit an application's access to an account. When an application requests a scope, the access token issued to the application is limited to the scope granted.

Use the following scope to initialize your Google API client:

https://www.googleapis.com/auth/chronicle-backstory

Python example

The following Python example demonstrates how to use the OAuth2 credentials and HTTP client using google.oauth2 and googleapiclient.

# Imports required for the sample - Google Auth and API Client Library Imports.
# Get these packages from https://pypi.org/project/google-api-python-client/ or run $ pip
# install google-api-python-client from your terminal
from google.oauth2 import service_account
from googleapiclient import _auth

SCOPES = ['https://www.googleapis.com/auth/chronicle-backstory']

# The apikeys-demo.json file contains the customer's OAuth 2 credentials.
# SERVICE_ACCOUNT_FILE is the full path to the apikeys-demo.json file
# ToDo: Replace this with the full path to your OAuth2 credentials
SERVICE_ACCOUNT_FILE = '/customer-keys/apikeys-demo.json'

# Create a credential using Google Developer Service Account Credential and Chronicle API
# Scope.
credentials = service_account.Credentials.from_service_account_file(SERVICE_ACCOUNT_FILE, scopes=SCOPES)

# Build an HTTP client to make authorized OAuth requests.
http_client = _auth.authorized_http(credentials)

# <your code continues here>

Chronicle API query limits

The Chronicle API enforces limits on the volume of requests that can be made by any one customer against the Chronicle platform. If you reach or exceed the query limit, the Chronicle API server returns HTTP 429 (RESOURCE_EXHAUSTED) to the caller. When developing applications for the Chronicle API, Chronicle recommends that you enforce rate limits within your system to avoid resource exhaustion. These limits apply to all of the Chronicle APIs, including the Search, Customer Management, and Tooling APIs.

The following limit for the Chronicle Customer Management API is being enforced and is measured in queries per second (QPS):

Chronicle API API Method Limit
Feed Management CreateFeed 1 QPS
GetFeed 1 QPS
ListFeeds 1 QPS
UpdateFeed 1 QPS
DeleteFeed 1 QPS

Python example using OAuth2 credentials and HTTP client

The following Python example demonstrates how to use the OAuth2 credentials and the HTTP client using google.oauth2 and googleapiclient.

# Imports required for the sample - Google Auth and API Client Library Imports.
# Get these packages from https://pypi.org/project/google-api-python-client/ or
# run $ pip install google-api-python-client from your terminal

from google.auth.transport import requests
from google.oauth2 import service_account

SCOPES = ['https://www.googleapis.com/auth/chronicle-backstory']

# The apikeys-demo.json file contains the customer's OAuth 2 credentials.
# SERVICE_ACCOUNT_FILE is the full path to the apikeys-demo.json file
# ToDo: Replace this with the full path to your OAuth2 credentials

SERVICE_ACCOUNT_FILE = '/customer-keys/apikeys-demo.json'

# Create a credential using Google Developer Service Account Credential and Chronicle # API Scope.

credentials = service_account.Credentials.from_service_account_file(SERVICE_ACCOUNT_FILE, scopes=SCOPES)

# Build an HTTP session to make authorized OAuth requests.

http_session = requests.AuthorizedSession(credentials)

# <your code continues here>

Feed Management API Reference

The Chronicle Feed Management API enables you to create and manage data feeds from a variety of different data sources to your Chronicle account. You can also configure Feed Management using the user interface. See here for more information.

CreateFeed

Creates a third party data feed in your Chronicle instance.

Request

https://backstory.googleapis.com/v1/feeds
Request Body

This example shows how to collect authentication logs from Duo Security.

{
  "display_name": "some feed name",
  "details": {
    "feedSourceType": "API",
    "logType": "DUO_AUTH",
    "duoAuthSettings": {
      "authentication": {
        "user": "ABCUSERNAMEDEF",
        "secret": "aBcS3cReTdEf"
      },
      "hostname": "api-abc123.duosecurity.com"
    },
    "namespace": "my-asset-namespace",
    "labels": [{
      "key": "my-ingestion-label-key",
      "value": "my-ingestion-label-value"
    }]
  }
}

Sample Request
https://backstory.googleapis.com/v1/feeds
{
  "display_name": "some feed name",
  "details": {
    "feedSourceType": "API",
    "logType": "DUO_AUTH",
    "duoAuthSettings": {
      "authentication": {
        "user": "ABCUSERNAMEDEF",
        "secret": "aBcS3cReTdEf"
      },
      "hostname": "api-abc123.duosecurity.com"
    }
  }
}
Sample Successful Response
{
 "name": "feeds/19e82867-ab6d-4955-b9c8-bd4aee189439",
 "display_name": "some feed name",
 "details": {
   "logType": "DUO_AUTH",
   "feedSourceType": "API",
   "duoAuthSettings": {
     "hostname": "api-abc123.duosecurity.com"
   }
 },
 "feedState": "ACTIVE"
}

If the response is unsuccessful, it returns an HTTP status code other than 200 (OK). Be sure to check the body of the response for details of the failure.

Asset namespace

To assign an asset namespace to all events that are ingested from a particular feed, set the "namespace" field within "details". The "namespace" field is a string.

Ingestion label

Ingestion labels are part of Unified Data Model metadata. They are repeated key and value pairs. To assign ingestion labels to all events that are ingested from a particular feed, set the "labels" field within "details". The "labels" field is an array of JSON objects with "key" and "value" fields.

DeleteFeed

Deletes a feed that was configured using the Chronicle Feed Management API.

Request

DELETE  https://backstory.googleapis.com/v1/feeds/{feedID}
Sample Request
DELETE https://backstory.googleapis.com/v1/feeds/01777371-b27b-44e7-8b2d-774302d7958f
Sample Response

If the operation is successful, DeleteFeed returns an empty response with an HTTP status code 200 (OK).

{}

EnableFeed

Enables an INACTIVE feed, which allows it to be executed.

Request
POST https://backstory.googleapis.com/v1/feeds/{feedID}:enable
Sample Request
POST https://backstory.googleapis.com/v1/feeds/3c9be423-b8c7-4cb7-a59e-024da8b30564:enable
Sample Response
{
 "name": "feeds/19e82867-ab6d-4955-b9c8-bd4aee189439",
 "display_name": "some feed name",
 "details": {
   "logType": "DUO_AUTH",
   "feedSourceType": "API",
   "duoAuthSettings": {
     "hostname": "api-abc123.duosecurity.com"
   }
 },
 "feedState": "ACTIVE"
}

DisableFeed

Disables a feed. A disabled feed has a status of INACTIVE. Disabled feeds will no longer fetch data.

Request
POST https://backstory.googleapis.com/v1/feeds/{feedID}:disable
Sample Request
POST https://backstory.googleapis.com/v1/feeds/3c9be423-b8c7-4cb7-a59e-024da8b30564:disable
Sample Response
{
 "name": "feeds/19e82867-ab6d-4955-b9c8-bd4aee189439",
 "display_name": "some feed name",
 "details": {
   "logType": "DUO_AUTH",
   "feedSourceType": "API",
   "duoAuthSettings": {
     "hostname": "api-abc123.duosecurity.com"
   }
 },
 "feedState": "INACTIVE"
}

GetFeed

Gets the details of the feed that was configured.

Request

https://backstory.googleapis.com/v1/feeds/{feedID}
Sample Request
https://backstory.googleapis.com/v1/feeds/01777371-b27b-44e7-8b2d-774302d7958f
Sample Response
{
 "name": "feeds/01777371-b27b-44e7-8b2d-774302d7958f",
 "display_name": "some feed name",
 "details": {
   "logType": "DUO_AUTH",
   "feedSourceType": "API",
   "duoAuthSettings": {
     "hostname": "api-abc123.duosecurity.com"
   }
 },
 "feedState": "ACTIVE"
}

ListFeeds

Retrieves all the feeds configured for a given Chronicle instance.

Request
https://backstory.googleapis.com/v1/feeds
Sample Request
https://backstory.googleapis.com/v1/feeds
Sample Response
{
 "feeds": [
   {
     "name": "feeds/19e82867-ab6d-4955-b9c8-bd4aee189439",
     "details": {
       "logType": "AZURE_AD_CONTEXT",
       "feedSourceType": "API",
       "azureAdContextSettings": {}
     },
     "feedState": "ACTIVE"
   },
   {
     "name": "feeds/cdc096a5-93a8-4854-94d9-c05cf0c14d47",
     "display_name": "some feed name",
     "details": {
       "logType": "PAN_PRISMA_CLOUD",
       "feedSourceType": "API",
       "panPrismaCloudSettings": {
         "hostname": "api2.prismacloud.io"
       }
     },
     "feedState": "ACTIVE"
   }
 ]
}

Read only feeds

There may be feeds returned from a ListFeeds request that have the field readOnly set to true. Read only feeds cannot be created, updated, or deleted.

Feeds are read-only for a few reasons. For example: * There exists certain feed source types that are not fully supported by Feed Management at the moment, and were created before the release of Feed Management. * There are certain specialized log types that are not available to every Chronicle user. If a feed exists with one of these types, it is considered read-only.

UpdateFeed

Updates the given feed with new details.

Request
PATCH https://backstory.googleapis.com/v1/feeds/{feedID}
Request Body

The following examples shows how to update a Duo Auth feed.

Sample Request
{
  "display_name": "my feed",
  "details": {
    "feedSourceType": "API",
    "logType": "DUO_AUTH",
    "duoAuthSettings": {
      "authentication": {
        "user": "ABCUSERNAMEDEF",
        "secret": "aBcS3cReTdEf"
      },
      "hostname": "api-abc123.duosecurity.com"
    }
  }
}
Sample Response
{
 "display_name": "my feed",
 "name": "feeds/19e82867-ab6d-4955-b9c8-bd4aee189439",
 "details": {
   "logType": "DUO_AUTH",
   "feedSourceType": "API",
   "duoAuthSettings": {
     "hostname": "api-abc123.duosecurity.com"
   }
 },
 "feedState": "ACTIVE"
}

Feed State

The "feedState" field can be found in the response message of most operations. "feedState" gives some insight into the current state of a feed.

feedState Description
"ACTIVE" Feed successfully created and will begin fetching data.
"INACTIVE" Feed has been disabled.
"IN_PROGRESS" Feed is currently attempting to fetch data. A feed will only have this status if it has not previously failed.
"COMPLETED" Feed has recently fetched data successfully.
"FAILED" Feed has failed and has not successfully fetched data since it failed. Mis-configuration is the typical cause of feed failure. Please see the "failureMsg" field for more information.

Failure Message

The "failureMsg" field can be found in the response message of most operations, but only for those feeds whose "feedState" is "FAILED". It provides more information regarding the nature of the failure, such as the response code returned from the relevant third party. Please refer to the documentation below for your particular feed type to understand how to correctly configure the feed.

Feed type

Chronicle supports many different types of feeds. These include feeds which gather data from either first or third party APIs, or from various cloud-based object stores.

Specifying feed type

When creating or editing a feed, it is necessary to specify the feed type. This is done by specifying two fields within the details portion of the request body for both CreateFeed and UpdateFeed. These fields are details.feedSourceType and details.logType.

Feed Source Type

Feed Source Type describes how data is collected. Valid values for Feed Source Type include:

details.feedSourceType Description
"API" Using a Chronicle supported API to gather data.
"GOOGLE_CLOUD_STORAGE" Ingest data from a Google Cloud Storage bucket.
"AMAZON_S3" Ingest data from an Amazon Simple Storage Service bucket.
"AMAZON_SQS" Ingest data from an Amazon Simple Queue Service queue whose entries point to files stored in S3.
"AZURE_BLOBSTORE" Ingest data from Azure Blob Storage.
"HTTP" Ingest data from files accessible by an HTTP(S) request. Note that this should not be used to interact with third-party APIs. Use the "API" Feed Source Type for APIs supported by Chronicle.

Log Type

Log Type is a label which describes the nature of the data being ingested. Chronicle supports many different Log Types, but there are some limitations.

API Log Types

When the Feed Source Type is "API", the Log Type corresponds to the particular API that Chronicle supports. Chronicle supports the following Log Types when collecting data using an API.

details.logType Feed Type
"ANOMALI_IOC" Anomali ThreatStream
"AZURE_AD" Azure Active Directory Sign-ins
"AZURE_AD_AUDIT" Azure Active Directory Audit
"AZURE_AD_CONTEXT" Azure Active Directory Organizational Context
"AZURE_MDM_INTUNE" Microsoft Intune
"CLOUD_PASSAGE" Cloud Passage Events
"CORTEX_XDR" Palo Alto Cortex XDR
"CS_DETECTS" CrowdStrike Detection Monitoring
"DUO_AUTH" Duo Auth
"DUO_USER_CONTEXT" Duo User Context
"FOX_IT_STIX" Fox-IT
"GCP_CLOUDIDENTITY_DEVICES" Google Cloud Identity Devices
"GCP_CLOUDIDENTITY_DEVICEUSERS" Google Cloud Identity Device Users
"IMPERVA_WAF" Imperva
"MICROSOFT_GRAPH_ALERT" Microsoft Graph API Alerts
"MICROSOFT_SECURITY_CENTER_ALERT" Microsoft Security Center
"MIMECAST_MAIL" Mimecast
"NETSKOPE_ALERT" Netskope Alerts
"OFFICE_365" Office 365 Management Activity
"OKTA" Okta
"OKTA_USER_CONTEXT" Okta User Context
"PAN_IOC" Palo Alto Autofocus
"PAN_PRISMA_CLOUD" Palo Alto Prisma Cloud
"PROOFPOINT_MAIL" Proofpoint Tap Alerts
"PROOFPOINT_ON_DEMAND" Proofpoint On Demand
"QUALYS_VM" Qualys VM
"RAPID7_INSIGHT" Rapid7 Insight
"RECORDED_FUTURE_IOC" Recorded Future
"RH_ISAC_IOC" RH-ISAC
"SALESFORCE" Salesforce
"SENTINELONE_ALERT" SentinelOne Alert
"SERVICENOW_CMDB" ServiceNow CMDB
"SYMANTEC_EVENT_EXPORT" Symantec Event Export
"THINKST_CANARY" Thinkst Canary
"THREATCONNECT_IOC" ThreatConnect
"WORKDAY" Workday
"WORKSPACE_ACTIVITY" Workspace Activities
"WORKSPACE_ALERTS" Workspace Alerts
"WORKSPACE_CHROMEOS" Workspace ChromeOS Devices
"WORKSPACE_GROUPS" Workspace Groups
"WORKSPACE_MOBILE" Workspace Mobile Devices
"WORKSPACE_PRIVILEGES" Workspace Privileges
"WORKSPACE_USERS" Workspace Users
Log Types for Feed Source Types other than API

Chronicle supports many different log types which are compatible with feeds that ingest data from cloud-based object stores. See the Feed Schema for instructions on retrieving the current list of compatible Log Types.

Feed configuration by type

Each feed type has its own requirements for which fields must be set in order to correctly create or update a feed.

Google Cloud Storage

Data source Ingest schedule detailts.feedSourceType details.logType
Google Cloud Storage Bucket Every 15 minutes "GOOGLE_CLOUD_STORAGE" See Feed Schema to get compatible log types.

Prerequisites

Before setting up a Google Cloud Storage feed, you must grant Chronicle access. You must add the email address 8911409095528497-0-account@partnercontent.gserviceaccount.com to the permissions of the relevant Google Cloud Storage object(s). Perform the following actions from the Cloud Storage section in the Google Cloud Console (console.cloud.google.com)

  • To grant read permission to a specific file, you can "Edit access" on that file and grant the above email "Reader" access. This can only be done if you have not enabled uniform bucket-level access.
  • To grant read permission to multiple files you must grant access at the bucket level. Specifically, you must add the above email as a principal to your storage bucket and grant it the IAM role of Storage Object Viewer.
  • If you configure the feed to delete source files (see below for how to do this), you must add the above email as a principal on your bucket and grant it the IAM role of Storage Object Admin.

Type-specific request fields

Field Required Description
details.gcsSettings.bucketUri Yes The URI which corresponds to the Google Cloud Storage bucket. The format is the same format used by gsutil to specify a resource.
details.gcsSettings.sourceType Yes The type of object indicated by bucketUri. See below for valid values.
details.gcsSettings.sourceDeletionOption Yes Whether to delete source files after they have been transferred to Chronicle. See below for valid values.

Sample CreateFeed request

{
 "details": {
   "feedSourceType": "GOOGLE_CLOUD_STORAGE",
   "logType": "LOGTYPE_YOU_WANT_TO_BRING",
   "gcsSettings": {
     "bucketUri": "gs://bucket/file",
     "sourceType": "FOLDERS_RECURSIVE",
     "sourceDeletionOption": "SOURCE_DELETION_NEVER"
   }
 }
}

Amazon S3

Data source Ingest schedule detailts.feedSourceType details.logType
Amazon Simple Storage Service Bucket Every 15 minutes "AMAZON_S3" See Feed Schema to get compatible log types.

Prerequisites

  1. Create an S3 bucket.
  2. Create a security key for programmatic access.

Type-specific request fields

Field Required Description
details.amazonS3Settings.s3Uri Yes The S3 URI to ingest.
details.amazonS3Settings.sourceType Yes The type of file indicated by the uri. See below for valid values to specify.
details.amazonS3Settings.sourceDeletionOption Yes Whether to delete source files after they have been transferred to Chronicle. See below for valid values.
details.amazonS3Settings.authentication.region Yes The region where the S3 bucket resides. See below for a list of regions.
details.amazonS3Settings.authentication.accessKeyId Yes This is the 20 character ID associated with your Amazon IAM account.
details.amazonS3Settings.authentication.secretAccessKey Yes This is the 40 character access key associated with your Amazon IAM account.

Sample CreateFeed request

{
 "details": {
   "feedSourceType": "AMAZON_S3",
   "logType": "LOGTYPE_YOU_WANT_TO_BRING",
   "amazonS3Settings": {
     "s3Uri": "s3://uri/to/file",
     "sourceType": "FILES",
     "sourceDeletionOption": "SOURCE_DELETION_NEVER",
     "authentication": {
       "region": "US_EAST_1",
       "accessKeyId": "AKIAIOSFODNN7EXAMPLE",
       "secretAccessKey": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
     },
   }
 }
}

Amazon SQS

Data source detailts.feedSourceType details.logType
Amazon Simple Storage Service Bucket that sends notifications to an Amazon Simple Queueing Service queue "AMAZON_SQS" See Feed Schema to get compatible log types.

Amazon supports SQS queues which contain notifications from a monitored S3 bucket and Chronicle is able to read these notification off of an SQS queue and pull the corresponding files out of the S3 bucket. This is effectively a "push-based" version of an Amazon S3 feed and can be leveraged for better throughput.

Prerequisites

  1. Create an S3 bucket.
  2. Create an SQS queue.
    • The queue must be a Standard queue, not a FIFO queue.
  3. Set up notifications on your S3 bucket to write to your SQS queue.
    • Be sure to attach an access policy.
  4. Create an access key which will be used to access both the SQS queue and the S3 bucket.

Type-specific request fields

Field Required Description
details.amazonSqsSettings.queue Yes The SQS queue name.
details.amazonSqsSettings.region Yes The region where the SQS queue and S3 bucket resides. See below for a list of regions.
details.amazonSqsSettings.accountNumber Yes The account number for the SQS queue and S3 bucket.
details.amazonSqsSettings.sourceDeletionOption Yes Whether to delete source files out of the S3 bucket after they have been transferred to Chronicle. See below for valid values.
details.amazonSqsSettings.authentication.sqsAccessKeySecretAuth.accessKeyId Yes This is the 20 character ID associated with your Amazon IAM account.
details.amazonSqsSettings.authentication.sqsAccessKeySecretAuth.secretAccessKey Yes This is the 40 character access key associated with your Amazon IAM account.
details.amazonSqsSettings.authentication.additionalS3AccessKeySecretAuth.accessKeyId No This is the 20 character ID associated with your Amazon IAM account. Only specify if using a different access key for the S3 bucket.
details.amazonSqsSettings.authentication.additionalS3AccessKeySecretAuth.secretAccessKey No This is the 40 character access key associated with your Amazon IAM account. Only specify if using a different access key for the S3 bucket.

Sample CreateFeed request

{
 "details": {
   "feedSourceType": "AMAZON_SQS",
   "logType": "LOGTYPE_YOU_WANT_TO_BRING",
   "amazonSqsSettings": {
     "queue": "cs-prod-canon-queue-07869dcf07be481f",
     "region": "US_EAST_1",
     "accountNumber": "123456789012",
     "sourceDeletionOption": "SOURCE_DELETION_NEVER"
     "authentication": {
       "sqsAccessKeySecretAuth": {
         "accessKeyId": "AKIAIOSFODNN7EXAMPLE",
         "secretAccessKey": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
       },
     },
   }
 }
}

Amazon S3 regions

AWS Region AWS Region code authentication.region
Asia Pacific (Mumbai) ap-south-1 "AP_SOUTH_1"
Asia Pacific (Seoul) ap-northeast-2 "AP_NORTHEAST_2"
Asia Pacific (Singapore) ap-southeast-1 "AP_SOUTHEAST_1"
Asia Pacific (Sydney) ap-southeast-2 "AP_SOUTHEAST_2"
Asia Pacific (Tokyo) ap-northeast-1 "AP_NORTHEAST_1"
AWS GovCloud (US-East) us-gov-east-1 "US_GOV_EAST_1"
AWS GovCloud (US-West) us-gov-west-1 "US_GOV_CLOUD"
Canada (Central) ca-central-1 "CA_CENTRAL_1"
China (Beijing) cn-north-1 "CN_NORTH_1"
China (Ningxia) cn-northwest-1 "CN_NORTHWEST_1"
Europe (Frankfurt) eu-central-1 "EU_CENTRAL_1"
Europe (Ireland) eu-west-1 "EU_WEST_1"
Europe (London) eu-west-2 "EU_WEST_2"
Europe (Paris) eu-west-3 "EU_WEST_3"
Europe (Stockholm) eu-north-1 "EU_NORTH_1"
South America (São Paulo) sa-east-1 "SA_EAST_1"
US East (N. Virginia) us-east-1 "US_EAST_1"
US East (Ohio) us-east-2 "US_EAST_2"
US West (N. California) us-west-1 "US_WEST_1"
US West (Oregon) us-west-2 "US_WEST_2"

Azure Blob Storage

Data source Ingest schedule detailts.feedSourceType details.logType
Microsoft Azure Blob Storage Container Every 15 minutes "AZURE_BLOBSTORE" See Feed Schema to get compatible log types.

Prerequisites

You will need either

Type-specific request fields

Field Required Description
details.azureBlobStoreSettings.azureUri Yes The URI pointing to a Azure Blob Storage blob or container
details.azureBlobStoreSettings.sourceType Yes The type of object indicated by the uri. See below for valid values to specify.
details.azureBlobStoreSettings.sourceDeletionOption Yes For the moment source file deletion is not supported in Azure, so this must be "SOURCE_DELETION_NEVER"
details.azureBlobStoreSettings.authentication.sharedKey No A shared key, a 512-bit random string in base64 encoding, authorized to access Azure Blob Storage. Required if not specifying an SAS Token.
details.azureBlobStoreSettings.authentication.sasToken No A Shared Access Signature authorized to access the Azure Blob Storage container.

Azure source type

When specifying an Azure URI you must also indicate the type of object is indicated by the URI.

details.sourceType Source type
FILES The URI points to a single blob that will be ingested with each execution of the feed.
FOLDERS_RECURSIVE The URI points to a Blob Storage container.

Sample CreateFeed request

{
 "details": {
   "feedSourceType": "AZURE_BLOBSTORE",
   "logType": "LOGTYPE_YOU_WANT_TO_BRING",
   "azureBlobStoreSettings": {
     "azureUri": "https://myaccount.blob.core.windows.net/logging",
     "sourceType": "FOLDERS_RECURSIVE",
     "sourceDeletionOption": "SOURCE_DELETION_NEVER"
     "authentication": {
       "sharedKey": "Uv38ByGCZU8WP18PmmIdcpVmx00QA3xNe7sEB9HixkmBhVrYaB0NhtHpHgAWeTnLZpTSxCKs0gigByk5SH9pmQ==",
     },
   }
 }
}

HTTP(S)

Data source Ingest schedule detailts.feedSourceType details.logType
Files available over the open internet via an HTTP request. Every 15 minutes "HTTP" See Feed Schema to get compatible log types.

WARNING: The HTTP type should not be used to gather data from an API. Please refer to the supported API feed types below.

Type-specific request fields

Field Required Description
details.httpSettings.uri Yes The URI pointing to a file or collection of files.
details.httpSettings.sourceType Yes The type of file indicated by the uri. See below for valid values to specify
details.httpSettings.sourceDeletionOption Yes Whether to delete source files after they have been transferred to Chronicle. See below for valid values.

Sample CreateFeed request

{
 "details": {
   "feedSourceType": "HTTP",
   "logType": "LOGTYPE_YOU_WANT_TO_BRING",
   "httpSettings": {
     "uri": "https://url.com/myfile",
     "sourceType": "FILES",
     "sourceDeletionOption": "SOURCE_DELETION_NEVER"
   }
 }
}

Anomali ThreatStream

Data source Ingest schedule details.feedSourceType details.logType
api.threatstream.com Every minute "API" "ANOMALI_IOC"

Type-specific request fields

Field Required Description
details.anomaliSettings.authentication.user Yes Username
details.anomaliSettings.authentication.secret Yes API key

Sample CreateFeed request

{
 "details": {
   "feedSourceType": "API",
   "logType": "ANOMALI_IOC",
   "anomaliSettings": {
     "authentication": {
       "user": "USERNAME",
       "secret": "APIKEY"
     },
   }
 }
}

Azure Active Directory Sign-ins

Data source Ingest schedule details.feedSourceType details.logType
graph.microsoft.com Every hour "API" "AZURE_AD"

Prerequisites

The user whose credentials are used to authenticate against the Microsoft Graph API to access sign-ins must have the permissions AuditLog.Read.All and Directory.Read.All.

Type-specific request fields

Field Required Description
details.azureAdSettings.authentication.clientId Yes OAuth client ID (a UUID)
details.azureAdSettings.authentication.clientSecret Yes OAuth client Secret
details.azureAdSettings.tenantId Yes Tenant ID (a UUID)
details.azureAdSettings.hostname No API Full Path, default value : "graph.microsoft.com/v1.0/auditLogs/signIns"

Sample CreateFeed request

{
 "details": {
   "feedSourceType": "API",
   "logType": "AZURE_AD",
   "azureAdSettings": {
     "authentication": {
       "clientId": "7ab79b26-f3ef-425c-9221-cf95a36f19b6",
       "clientSecret": "clientSecret",
     }
     "tenantId": "0fc279f9-fe30-41be-97d3-abe1d7681418",
     "hostname": "graph.microsoft.com/v1.0/auditLogs/signIns",
   }
 }
}

Azure Active Directory Audit

Data source Ingest schedule details.feedSourceType details.logType
graph.microsoft.com Every minute "API" "AZURE_AD_AUDIT"

Prerequisites

The user whose credentials are used to authenticate against the Microsoft Graph API to access directory audits must have the permissions AuditLog.Read.All and Directory.Read.All.

Type-specific request fields

Field Required Description
details.azureAdAuditSettings.authentication.clientId Yes OAuth client ID (a UUID)
details.azureAdAuditSettings.authentication.clientSecret Yes OAuth client Secret
details.azureAdAuditSettings.tenantId Yes Tenant ID (a UUID)
details.azureAdAuditSettings.hostname No API Full Path, default value : "graph.microsoft.com/v1.0/auditLogs/directoryAudits"

Sample CreateFeed request

{
 "details": {
   "feedSourceType": "API",
   "logType": "AZURE_AD_AUDIT",
   "azureAdAuditSettings": {
     "authentication": {
       "clientId": "7ab79b26-f3ef-425c-9221-cf95a36f19b6",
       "clientSecret": "clientSecret",
     }
     "tenantId": "0fc279f9-fe30-41be-97d3-abe1d7681418",
     "hostname": "graph.microsoft.com/v1.0/auditLogs/directoryAudits",
   }
 }
}

Azure Active Directory Organizational Context

Data source Ingest schedule details.feedSourceType details.logType
graph.microsoft.com Every 24 hours "API" "AZURE_AD_AUDIT"

Prerequisites

The user whose credentials are used to authenticate against the Microsoft Graph API to access organizational context must have the permissions Directory.Read.All.

Type-specific request fields

Field Required Description
details.azureAdContextSettings.authentication.clientId Yes OAuth client ID (a UUID)
details.azureAdContextSettings.authentication.clientSecret Yes OAuth client secret
details.azureAdContextSettings.tenantId Yes Tenant ID (a UUID)
details.azureAdContextSettings.retrieveDevices No Whether to retrieve device information
details.azureAdContextSettings.retrieveGroups No Whether to retrieve user group information
details.azureAdContextSettings.hostname No API Full Path, default value : "graph.microsoft.com/beta"

Sample CreateFeed request

{
 "details": {
   "feedSourceType": "API",
   "logType": "AZURE_AD_CONTEXT",
   "azureAdContextSettings": {
     "authentication": {
       "clientId": "7ab79b26-f3ef-425c-9221-cf95a36f19b6",
       "clientSecret": "clientSecret",
     }
     "tenantId": "0fc279f9-fe30-41be-97d3-abe1d7681418",
     "retrieveDevices": false,
     "retrieveGroups": false,
     "hostname": "graph.microsoft.com/beta",
   }
 }
}

CrowdStrike Detection Monitoring

Data source Ingest schedule details.feedSourceType details.logType
api.crowdstrike.com Every minute "API" "CS_DETECTS"

Type-specific request fields

Field Required Description
details.crowdstrikeDetectsSettings.authentication.clientId Yes OAuth Client ID
details.crowdstrikeDetectsSettings.authentication.clientSecret Yes OAuth Client Secret
details.crowdstrikeDetectsSettings.authentication.tokenEndpoint Yes Authentication URL
details.crowdstrikeDetectsSettings.hostname Yes API Endpoint URL

Sample CreateFeed request

{
  "details": {
    "feedSourceType": "API",
    "logType": "CS_DETECTS",
    "crowdstrikeDetectsSettings": {
      "authentication": {
          "clientId": "CLIENT ID",
          "clientSecret": "CLIENT SECRET",
          "tokenEndpoint": "https://api.us-2.crowdstrike.com/oauth2/token"
      },
      "hostname": "api.crowdstrike.com"
    }       
  }
}

Steps to enable Crowdstrike Feed

Create a CrowdStrike API Client
  1. Within the CrowdStrike application, create an API client by navigating to Support and resources > API clients and keys.

    Select API clients and keys

  2. Create a new API Client with API scopes granting permission to Read Detections.

    Configure API client

  3. Record the values for: Base URL + Client ID + Client Secret. You will need these to set up the Feed in Chronicle.

    New OAuth2 API client

Setup the Chronicle feed
  1. Launch Chronicle, then select the Settings menu.
  2. Select Feeds in the left pane and click Add New.
  3. Select the Third Party API Source Type and Crowdstrike Detection Monitoring log type.

    Set CS_DETECTS source type and log type

  4. Fill in the requested parameters gathered earlier from CrowdStrike and click Submit.

    Set CS_DETECTS input parameters

The feed will begin to retrieve all detections from the CrowdStrike instance in chronological order. Detections older than 6 months will be dropped. After the backfill is complete, which can take some time depending on the number of detections in CrowdStrike, the feed checks for new detections every 5 minutes.

Microsoft Intune

Sample CreateFeed request

{
   "details": {
     "feedSourceType": "API",
     "logType": "AZURE_MDM_INTUNE",
     "azureMdmIntuneSettings": {
       "authentication": {
         "clientId": "7ab79b26-f3ef-425c-9221-cf95a36f19b6",
         "clientSecret": "clientSecret",
       }
       "tenantId": "0fc279f9-fe30-41be-97d3-abe1d7681418",
       "hostname": "graph.microsoft.com/beta/deviceManagement/auditEvents",
     }
   }
}

Cloud Passage Events

Sample CreateFeed request

{
   "details": {
     "feedSourceType": "API",
     "logType": "CLOUD_PASSAGE",
     "cloudPassageSettings": {
       "authentication": {
         "user": "api_key_id",
         "secret": "api_key_secret",
       }
       "eventTypes": [
         "fim_target_integrity_changed",
         "lids_rule_failed",
         "sca_rule_failed"
       ],
     }
   }
}

Palo Alto Cortex XDR

Sample CreateFeed request

{
   "details": {
     "feedSourceType": "API",
     "logType": "CORTEX_XDR",
     "cortexXdrSettings": {
       "authentication": {
         "headerKeyValues": [{
            "key": "Authorization"
            "value": "api_key"
         },
         {
            "key": "x-xdr-auth-id"
            "value": "api_key_id"
         }
         ],
       },
       "hostname": "api-abcd.xdr.ab.paloaltonetworks.com",
       "endpoint": "incidents"
     }
   }
}

Duo Auth

Sample CreateFeed request

{
   "details": {
     "feedSourceType": "API",
     "logType": "DUO_AUTH",
     "duoAuthSettings": {
       "authentication": {
         "user": "USERNAME",
         "secret": "SECRET"
       },
       "hostname": "api-mytenant.duosecurity.com"
     }
   }
}

Duo User Context

Sample CreateFeed request

{
   "details": {
     "feedSourceType": "API",
     "logType": "DUO_USER_CONTEXT",
     "duoUserContextSettings": {
       "authentication": {
         "user": "USERNAME",
         "secret": "SECRET"
       },
       "hostname": "api-mytenant.duosecurity.com"
     }
   }
}

Fox-IT

Sample CreateFeed request

{
   "details": {
     "feedSourceType": "API",
     "logType": "FOX_IT_STIX",
     "foxItStixSettings": {
       "authentication": {
         "user": "USERNAME",
         "secret": "SECRET"
       },
       "ssl": {
         "sslCertificate": "<cert>",
         "encodedPrivateKey": "key"
       }
       "pollServiceURI": "https://stix.fox-it.com/services/poll",
       "collection": "mycollection"
     }
   }
}

Google Cloud Identity Devices

Sample CreateFeed request

{
 "details": {
   "feedSourceType": "API",
   "logType": "GCP_CLOUDIDENTITY_DEVICES",
   "googleCloudIdentityDevicesSettings": {
     "authentication": {
       "tokenEndPoint": "jwt_token_uri",
       "claims": {
         "issuer": "jwt_client_email",
         "subject": "user_email",
         "audience": "jwt_token_uri"
       },
       "rsCredentials": {
         "private_key": "privatekey"
       }
     },
     "apiVersion": "v1",
   }
 }
}

Google Cloud Identity Device Users

Sample CreateFeed request

{
 "details": {
   "feedSourceType": "API",
   "logType": "GCP_CLOUDIDENTITY_DEVICEUSERS",
   "googleCloudIdentityDeviceUsersSettings": {
     "authentication": {
       "tokenEndPoint": "jwt_token_uri",
       "claims": {
         "issuer": "jwt_client_email",
         "subject": "user_email",
         "audience": "jwt_token_uri"
       },
       "rsCredentials": {
         "private_key": "privatekey"
       }
     },
   }
 }
}

Imperva

Sample CreateFeed request

{
   "details": {
     "feedSourceType": "API",
     "logType": "IMPERVA_WAF",
     "impervaWafSettings": {
       "authentication": {
         "headerKeyValues": [{
            "key": "key"
            "value": "value"
         }],
       }
     }
   }
}

Microsoft Graph API Alerts

Sample CreateFeed request

{
   "details": {
     "feedSourceType": "API",
     "logType": "MICROSOFT_GRAPH_ALERT",
     "microsoftGraphAlertSettings": {
       "authentication": {
         "clientId": "7ab79b26-f3ef-425c-9221-cf95a36f19b6",
         "clientSecret": "clientSecret",
       }
       "tenantId": "0fc279f9-fe30-41be-97d3-abe1d7681418",
       "hostname": "graph.microsoft.com/v1.0/security/alerts",
     }
   }
}

Microsoft Security Center

Sample CreateFeed request

{
   "details": {
     "feedSourceType": "API",
     "logType": "MICROSOFT_SECURITY_CENTER_ALERT",
     "microsoftSecurityCenterAlertSettings": {
       "authentication": {
         "clientId": "7ab79b26-f3ef-425c-9221-cf95a36f19b6",
         "clientSecret": "clientSecret",
       }
       "tenantId": "0fc279f9-fe30-41be-97d3-abe1d7681418",
       "subscriptionId": "0fc279f9-fe30-41be-97d3-abe1d7681418",
       "hostname": "management.azure.com",
     }
   }
}

Mimecast

Sample CreateFeed request

MIMECAST_MAIL

{
   "details": {
     "feedSourceType": "API",
     "logType": "MIMECAST_MAIL",
     "mimecastMailSettings": {
       "authentication": {
         "headerKeyValues": [
           {
             "key": "access_key",
             "value": "ACCESS_KEY"
           },
           {
             "key": "app_id",
             "value": "APP_ID"
           },
           {
             "key": "app_key",
             "value": "APP_KEY"
           },
           {
             "key": "secret_key",
             "value": "SECRET_KEY"
           }
         ]
       },
       "hostname": "xx-api.mimecast.com"
     }
   }
}

Netskope Alerts

Sample CreateFeed request

{
   "details": {
     "feedSourceType": "API",
     "logType": "NETSKOPE_ALERT",
     "netskopeAlertSettings": {
       "authentication": {
         "user": "user",
         "secret": "secret"
       },
       "hostname": "hostname",
       "feedName": "feedname"
     }
   }
}

Office 365 Management Activity

Data source Ingest schedule details.feedSourceType details.logType
manage.office.com Every minute "API" "OFFICE_365"

Prerequisites

The user whose credentials are used to authenticate against the API must have the permissions ActivityFeed.Read. If ingesting DLP data then the permission ActivityFeed.ReadDlp must be specified.

Type-specific request fields

Field Required Description
details.office365Settings.authentication.clientId Yes OAuth client ID (a UUID)
details.office365Settings.authentication.clientSecret Yes OAuth client secret
details.office365Settings.tenantId Yes Tenant ID (a UUID)
details.office365Settings.contentType Yes The type of logs to fetch. See below to see the valid values for contentType.
details.office365Settings.hostname No API Full Path, default value: "manage.office.com/api/v1.0"

Office 365 Content Type

Feed Source Type describes how data are collected. Valid values for Feed Source Type include:

details.office365Settings.contentType Description
"AUDIT_AZURE_ACTIVE_DIRECTORY" Azure active directory audit logs.
"AUDIT_EXCHANGE" Azure exchange audit logs.
"AUDIT_SHARE_POINT" Azure share point audit logs.
"AUDIT_GENERAL" All other workloads not included in other Audit content types.
"DLP_ALL" DLP events only for all workloads.

Sample CreateFeed request

{
   "details": {
     "feedSourceType": "API",
     "logType": "OFFICE_365",
     "office365Settings": {
       "authentication": {
         "clientId": "7ab79b26-f3ef-425c-9221-cf95a36f19b6",
         "clientSecret", "clientSecret",
       },
       "tenantId": "0fc279f9-fe30-41be-97d3-abe1d7681418"",
       "contentType": "AUDIT_AZURE_ACTIVE_DIRECTORY",
       "hostname": "manage.office.com/api/v1.0",
     }
   }
}

Okta

Sample CreateFeed request

{
   "details": {
     "feedSourceType": "API",
     "logType": "OKTA",
     "oktaSettings": {
       "authentication": {
         "headerKeyValues": [{
            "key": "Authorization",
            "value": "APITOKEN"
          }]
       },
       "hostname": "hostname"
     }
   }
}

Okta User Context

Sample CreateFeed request

managerIdReferenceField is required when you use a non-Okta ID to reference managers. It should be a JSON field path pointing to the field that contains the manager ID in the result of a call to the "users" Okta API.

{
   "details": {
     "feedSourceType": "API",
     "logType": "OKTA_USER_CONTEXT",
     "oktaSettings": {
       "authentication": {
         "headerKeyValues": [{
            "key": "Authorization",
            "value": "APITOKEN"
          }]
       },
       "hostname": "hostname",
       "managerIdReferenceField": "fooId"
     }
   }
}

Palo Alto Autofocus

Sample CreateFeed request

{
   "details": {
     "feedSourceType": "API",
     "logType": "PAN_IOC",
     "panIocSettings": {
       "authentication": {
         "headerKeyValues": [{
            "key": "key"
            "value": "value"
         }],
       }
       "feedId": "ID",
       "feed": "feed"
     }
   }
}

Palo Alto Prisma Cloud

Sample CreateFeed request

{
   "details": {
     "feedSourceType": "API",
     "logType": "PAN_PRISMA_CLOUD",
     "panPrismaCloudSettings": {
       "authentication": {
         "user": "user",
         "password": "password"
       },
       "hostname": "api2.prismacloud.io"
     }
   }
}

Proofpoint Tap Alerts

Sample CreateFeed request

{
   "details": {
     "feedSourceType": "API",
     "logType": "PROOFPOINT_MAIL",
     "proofpointMailSettings": {
       "authentication": {
         "user": "user",
         "secret": "secret"
       }
     }
   }
}

Proofpoint On Demand

Sample CreateFeed request

{
   "details": {
     "feedSourceType": "API",
     "logType": "PROOFPOINT_ON_DEMAND",
     "proofpointOnDemandSettings": {
       "authentication": {
         "user": "user",
         "secret": "secret"
       },
       "clusterId": "ID"
     }
   }
}

Qualys VM

Sample CreateFeed request

{
   "details": {
     "feedSourceType": "API",
     "logType": "QUALYS_VM",
     "qualysVmSettings": {
       "authentication": {
         "user": "USERNAME",
         "secret": "PASSWORD"
       },
       "hostname": "qualysapi.qualys.com/api/2.0/fo/asset/host/?action=list"
     }
   }
}

Rapid7 Insight

Sample CreateFeed request

The endpoint for Rapid7 should be either "vulnerabilities" or "assets".

{
   "details": {
     "feedSourceType": "API",
     "logType": "RAPID7_INSIGHT",
     "rapid7InsightSettings": {
       "authentication": {
         "headerKeyValues": [{
            "key": "X-Api-Key",
            "value": "APIKEY"
         }],
       },
       "endpoint": "assets"
       "hostname": "us.api.insight.rapid7.com"
     }
   }
}

Recorded Future

Sample CreateFeed request

{
   "details": {
     "feedSourceType": "API",
     "logType": "RECORDED_FUTURE_IOC",
     "recordedFutureIocSettings": {
       "authentication": {
         "user": "user",
         "secret": "secret"
       },
     }
   }
}

RH-ISAC

Sample CreateFeed request

{
   "details": {
     "feedSourceType": "API",
     "logType": "RH_ISAC_IOC",
     "rhIsacIocSettings": {
       "authentication": {
         "tokenEndPoint": "endpoint",
         "clientId": "clientId",
         "clientSecret": "clientSecret"
       }
     }
   }
}

Salesforce

Sample CreateFeed request

{
   "details": {
     "feedSourceType": "API",
     "logType": "SALESFORCE",
     "salesforceSettings": {
       "authentication": {
         "tokenEndpoint": "endpoint",
         "clientId": "clientId",
         "clientSecret": "clientSecret",
         "user": "user",
         "password": "password"
       },
       "hostname": "hostname"
     }
   }
}

SentinelOne Alert

Sample CreateFeed request

{
   "details": {
     "feedSourceType": "API",
     "logType": "SENTINELONE_ALERT",
     "sentineloneAlertSettings": {
       "authentication": {
         "headerKeyValues": [{
            "key": "Authorization",
            "value": "ApiToken API_TOKEN"
          }]
       },
       "hostname": "hostname"
     }
   }
}

ServiceNow CMDB

Sample CreateFeed request

{
   "details": {
     "feedSourceType": "API",
     "logType": "SERVICENOW_CMDB",
     "servicenowCmdbSettings": {
       "authentication": {
         "user": "user",
         "secret": "secret"
       },
       "hostname": "hostname",
       "feedname": "feedname"
     }
   }
}

Symantec Event Export

Sample CreateFeed request

{
   "details": {
     "feedSourceType": "API",
     "logType": "SYMANTEC_EVENT_EXPORT",
     "symantecEventExportSettings ": {
       "authentication": {
         "tokenEndPoint": "REFRESH TOKEN URI",
         "clientId": "CLIENT ID",
         "clientSecret": "CLIENT SECRET",
         "refreshToken": "REFRESH TOKEN",
       }
     }
   }
}

Thinkst Canary

Sample CreateFeed request

{
   "details": {
     "feedSourceType": "API",
     "logType": "THINKST_CANARY",
     "thinkstCanarySettings": {
       "authentication": {
         "user": "user",
         "secret": "secret"
       },
       "hostname": "hostname"
     }
   }
}

ThreatConnect

Sample CreateFeed request

{
   "details": {
     "feedSourceType": "API",
     "logType": "THREATCONNECT_IOC",
     "threatConnectIocSettings": {
       "authentication": {
         "user": "user",
         "secret": "secret"
       },
       "hostname": "hostname",
       "owners": [{
         "owner"
       }]
     }
   }
}

Workday

Sample CreateFeed request

{
   "details": {
     "feedSourceType": "API",
     "logType": "WORKDAY",
     "workdaySettings": {
       "authentication": {
         "user": "ClientID",
         "secret": "ClientSecret"
       },
       "hostname": "hostname",
       "tenantId": "ID"
     }
   }
}

Workspace Activities

Data source Ingest schedule details.feedSourceType details.logType
admin.googleapis.com Every hour "API" "WORKSPACE_ACTIVITY"

Prerequisites

In order for Chronicle to ingest Workspace activities you must

  1. Enable the Admin SDK API on your Google Cloud project.
  2. Create a Service Account which will be used to authenticate against the Admin API.
  3. Generate a JSON key for the Service Account.
  4. Create a domain-wide delegation for the Service Account with the following OAuth scope:
    • https://www.googleapis.com/auth/admin.reports.audit.readonly
  5. Create a Workspace user and assign it an admin role which includes the "Reports" admin privilege, or create a custom role which includes that privilege.
  6. Locate your Workspace customer ID.

Type-specific request fields

Field Required Description
details.workspaceActivitySettings.authentication.tokenEndpoint Yes The value of the token_uri field in the JSON key for the service account created to access the admin API.
details.workspaceActivitySettings.authentication.claims.issuer Yes The value of the client_email field in the JSON key for the service account created to access the admin API.
details.workspaceActivitySettings.authentication.claims.subject Yes The email address of the Workspace admin user with "Reports" privilege.
details.workspaceActivitySettings.authentication.claims.audience Yes The value of the token_uri field in the JSON key for the service account created to access the admin API.
details.workspaceActivitySettings.authentication.rsCredentials.privateKey Yes The value of the private_key field in the JSON key for the service account created to access the admin API. Note that literal newline characters (\n) should be replaced with carriage returns. Also note that the field name is "rsCredentials", and not "rsaCredentials".
details.workspaceActivitySettings.workspaceCustomerId Yes The Workspace customer ID. Note that the customer ID must have a leading 'C' character. The customer ID may appear differently depending on where in the Google admin console it is found. If the customer ID you have does not have a leading 'C' then prepend what you have with a 'C'.
details.workspaceActivitySettings.applications Yes The Workspace applications to gather activities for. See below for valid values.

Workspace applications

Activities are associated with one or more applications. The applications that Chronicle supports include the following.

details.workspaceActivitySettings.applications Description
"access_transparency" Access Transparency log events
"admin" Admin log events
"calendar" Calendar log events
"chat" Chat log events
"drive" Drive log events
"gcp" Google Cloud Platform activity events
"gplus" Currents log events
"groups" Groups log events
"groups_enterprise" Groups Enterprise log events
"jamboard" Jamboard log events
"login" User log events
"meet" Meet log events
"mobile" Device log events
"rules" Rule log events (beta)
"saml" SAML log events
"token" OAuth log events
"user_accounts" User log events
"context_aware_access" Context-Aware Access log events
"chrome" Chrome log events
"data_studio" Looker Studio log events
"keep" Keep log events

Sample CreateFeed request

{
 "details": {
   "feedSourceType": "API",
   "logType": "WORKSPACE_ACTIVITY",
   "workspaceActivitySettings": {
     "authentication": {
       "tokenEndpoint": "https://accounts.google.com/o/oauth2/token",
       "claims": {
         "issuer": "service-account@project.iam.gserviceaccount.com",
         "subject": "user@domain.com",
         "audience": "https://accounts.google.com/o/oauth2/token"
       },
       "rsCredentials": {
         "privateKey": "-----BEGIN PRIVATE KEY-----
MIIEvQIBADANBgkqhkiG9w0BAQE...o/gyVcgdkPBHC+sLG8g702Q=
         -----END PRIVATE KEY-----"
       },
     },
     "workspaceCustomerId": "C1e2x3ample",
     "applications": [
       "admin",
       "groups",
       "mobile"
     ],
   }
 }
}

Workspace Alerts

Data source Ingest schedule details.feedSourceType details.logType
alertcenter.googleapis.com Every hour "API" "WORKSPACE_ALERTS"

Prerequisites

For Chronicle to ingest Workspace alerts, complete the following steps:

  1. Enable the Alert Center API on your Google Cloud project.
  2. Create a Service Account which will be used to authenticate against the Alert Center API.
  3. Generate a JSON key for the Service Account.
  4. Create a domain-wide delegation for the Service Account with the following OAuth scope:
    • https://www.googleapis.com/auth/apps.alerts
  5. Create a Workspace user and assign it an admin role which includes Alert Center view access, or create a custom role which includes that privilege.
  6. Locate your Workspace customer ID.

Type-specific request fields

Field Required Description
details.workspaceAlertsSettings.authentication.tokenEndpoint Yes The value of the token_uri field in the JSON key for the service account created to access the admin API.
details.workspaceAlertsSettings.authentication.claims.issuer Yes The value of the client_email field in the JSON key for the service account created to access the admin API.
details.workspaceAlertsSettings.authentication.claims.subject Yes The email address of the Workspace admin user with Alert Center view access.
details.workspaceAlertsSettings.authentication.claims.audience Yes The value of the token_uri field in the JSON key for the service account created to access the admin API.
details.workspaceAlertsSettings.authentication.rsCredentials.privateKey Yes The value of the private_key field in the JSON key for the service account created to access the Alert Center API. Note that literal newline characters (\n) should be replaced with carriage returns. Also note that the field name is "rsCredentials", and not "rsaCredentials".
details.workspaceAlertsSettings.workspaceCustomerId Yes The Workspace customer ID. Note that the customer ID must not have a leading 'C' character. The customer ID may appear differently depending on where in the Google admin console it is found. If the customer ID you have has a leading 'C' then remove it before including in your request.

Sample CreateFeed request

{
 "details": {
   "feedSourceType": "API",
   "logType": "WORKSPACE_ALERTS",
   "workspaceAlertsSettings": {
     "authentication": {
       "tokenEndpoint": "https://accounts.google.com/o/oauth2/token",
       "claims": {
         "issuer": "service-account@project.iam.gserviceaccount.com",
         "subject": "user@domain.com",
         "audience": "https://accounts.google.com/o/oauth2/token"
       },
       "rsCredentials": {
         "privateKey": "-----BEGIN PRIVATE KEY-----
MIIEvQIBADANBgkqhkiG9w0BAQE...o/gyVcgdkPBHC+sLG8g702Q=
         -----END PRIVATE KEY-----"
       },
     },
     "workspaceCustomerId": "1e2x3ample",
   }
 }
}

Workspace ChromeOS Devices

Data source Ingest schedule details.feedSourceType details.logType
admin.googleapis.com Every 24 hours "API" "WORKSPACE_CHROMEOS"

Prerequisites

For Chronicle to ingest Workspace ChromeOS devices, complete the following steps:

  1. Enable the Admin SDK API on your Google Cloud project.
  2. Create a Service Account which will be used to authenticate against the Admin API.
  3. Generate a JSON key for the Service Account.
  4. Create a domain-wide delegation for the Service Account with the following OAuth scope:
    • https://www.googleapis.com/auth/admin.directory.device.chromeos.readonly
  5. Create a Workspace user and assign it an admin role which includes Chrome Management Settings access, or create a custom role which includes that privilege.
  6. Locate your Workspace customer ID.

Type-specific request fields

Field Required Description
details.workspaceChromeOsSettings.authentication.tokenEndpoint Yes The value of the token_uri field in the JSON key for the service account created to access the admin API.
details.workspaceChromeOsSettings.authentication.claims.issuer Yes The value of the client_email field in the JSON key for the service account created to access the admin API.
details.workspaceChromeOsSettings.authentication.claims.subject Yes The email address of the Workspace admin user with "Reports" privilege.
details.workspaceChromeOsSettings.authentication.claims.audience Yes The value of the token_uri field in the JSON key for the service account created to access the admin API.
details.workspaceChromeOsSettings.authentication.rsCredentials.privateKey Yes The value of the private_key field in the JSON key for the service account created to access the admin API. Note that literal newline characters (\n) should be replaced with carriage returns. Note that the field name is "rsCredentials", and not "rsaCredentials".
details.workspaceChromeOsSettings.workspaceCustomerId Yes The Workspace customer ID. Note that the customer ID must have a leading 'C' character. The customer ID may appear differently depending on where in the Google admin console it is found. If the customer ID you have does not have a leading 'C' then prepend what you have with a 'C'.

Sample CreateFeed request

{
 "details": {
   "feedSourceType": "API",
   "logType": "WORKSPACE_CHROMEOS",
   "workspaceChromeOsSettings": {
     "authentication": {
       "tokenEndpoint": "https://accounts.google.com/o/oauth2/token",
       "claims": {
         "issuer": "service-account@project.iam.gserviceaccount.com",
         "subject": "user@domain.com",
         "audience": "https://accounts.google.com/o/oauth2/token"
       },
       "rsCredentials": {
         "privateKey": "-----BEGIN PRIVATE KEY-----
MIIEvQIBADANBgkqhkiG9w0BAQE...o/gyVcgdkPBHC+sLG8g702Q=
         -----END PRIVATE KEY-----"
       },
     },
     "workspaceCustomerId": "C1e2x3ample",
   }
 }
}

Workspace Groups

Data source Ingest schedule details.feedSourceType details.logType
admin.googleapis.com Every 24 hours "API" "WORKSPACE_GROUPS"

Prerequisites

For Chronicle to ingest Workspace ChromeOS devices, complete the following steps:

  1. Enable the Admin SDK API on your Google Cloud project.
  2. Create a Service Account which will be used to authenticate against the Admin API.
  3. Generate a JSON key for the Service Account.
  4. Create a domain-wide delegation for the Service Account with the following OAuth scope:
    • https://www.googleapis.com/auth/admin.directory.group.readonly
  5. Create a Workspace user and assign it an admin role which includes Admin API Group read privileges, or create a custom role which includes that privilege.
  6. Locate your Workspace customer ID.

Type-specific request fields

Field Required Description
details.workspaceGroupsSettings.authentication.tokenEndpoint Yes The value of the token_uri field in the JSON key for the service account created to access the admin API.
details.workspaceGroupsSettings.authentication.claims.issuer Yes The value of the client_email field in the JSON key for the service account created to access the admin API.
details.workspaceGroupsSettings.authentication.claims.subject Yes The email address of the Workspace admin user with the Admin API Group read privilege.
details.workspaceGroupsSettings.authentication.claims.audience Yes The value of the token_uri field in the JSON key for the service account created to access the admin API.
details.workspaceGroupsSettings.authentication.rsCredentials.privateKey Yes The value of the private_key field in the JSON key for the service account created to access the admin API. Note that literal newline characters (\n) should be replaced with carriage returns. Note that the field name is "rsCredentials", and not "rsaCredentials".
details.workspaceGroupsSettings.workspaceCustomerId Yes The Workspace customer ID. Note that the customer ID must have a leading 'C' character. The customer ID may appear differently depending on where in the Google admin console it is found. If the customer ID you have does not have a leading 'C' then prepend what you have with a 'C'.

Sample CreateFeed request

{
 "details": {
   "feedSourceType": "API",
   "logType": "WORKSPACE_GROUPS",
   "workspaceGroupsSettings": {
     "authentication": {
       "tokenEndpoint": "https://accounts.google.com/o/oauth2/token",
       "claims": {
         "issuer": "service-account@project.iam.gserviceaccount.com",
         "subject": "user@domain.com",
         "audience": "https://accounts.google.com/o/oauth2/token"
       },
       "rsCredentials": {
         "privateKey": "-----BEGIN PRIVATE KEY-----
MIIEvQIBADANBgkqhkiG9w0BAQE...o/gyVcgdkPBHC+sLG8g702Q=
         -----END PRIVATE KEY-----"
       },
     },
     "workspaceCustomerId": "C1e2x3ample",
   }
 }
}

Workspace Mobile Devices

Data source Ingest schedule details.feedSourceType details.logType
admin.googleapis.com Every 24 hours "API" "WORKSPACE_GROUPS"

Prerequisites

In order for Chronicle to ingest Workspace ChromeOS devices, complete the following steps:

  1. Enable the Admin SDK API on your Google Cloud project.
  2. Create a Service Account which will be used to authenticate against the Admin API.
  3. Generate a JSON key for the Service Account.
  4. Create a domain-wide delegation for the Service Account with the following OAuth scope:
    • https://www.googleapis.com/auth/admin.directory.group.readonly
  5. Create a Workspace user and assign it an admin role which includes Admin API Group read privileges, or create a custom role which includes that privilege.
  6. Locate your Workspace customer ID.

Type-specific request fields

Field Required Description
details.workspaceGroupsSettings.authentication.tokenEndpoint Yes The value of the token_uri field in the JSON key for the service account created to access the admin API.
details.workspaceGroupsSettings.authentication.claims.issuer Yes The value of the client_email field in the JSON key for the service account created to access the admin API.
details.workspaceGroupsSettings.authentication.claims.subject Yes The email address of the Workspace admin user with the Admin API Group read privilege.
details.workspaceGroupsSettings.authentication.claims.audience Yes The value of the token_uri field in the JSON key for the service account created to access the admin API.
details.workspaceGroupsSettings.authentication.rsCredentials.privateKey Yes The value of the private_key field in the JSON key for the service account created to access the admin API. Note that literal newline characters (\n) should be replaced with carriage returns. Note that the field name is "rsCredentials", and not "rsaCredentials".
details.workspaceGroupsSettings.workspaceCustomerId Yes The Workspace customer ID. Note that the customer ID must have a leading 'C' character. The customer ID may appear differently depending on where in the Google admin console it is found. If the customer ID you have does not have a leading 'C' then prepend what you have with a 'C'.

Sample CreateFeed request

{
 "details": {
   "feedSourceType": "API",
   "logType": "WORKSPACE_MOBILE",
   "workspaceMobileSettings": {
     "authentication": {
       "tokenEndpoint": "https://accounts.google.com/o/oauth2/token",
       "claims": {
         "issuer": "service-account@project.iam.gserviceaccount.com",
         "subject": "user@domain.com",
         "audience": "https://accounts.google.com/o/oauth2/token"
       },
       "rsCredentials": {
         "privateKey": "-----BEGIN PRIVATE KEY-----
MIIEvQIBADANBgkqhkiG9w0BAQE...o/gyVcgdkPBHC+sLG8g702Q=
         -----END PRIVATE KEY-----"
       },
     },
     "workspaceCustomerId": "C1e2x3ample",
   }
 }
}

Workspace Privileges

Data source Ingest schedule details.feedSourceType details.logType
admin.googleapis.com Every 24 hours "API" "WORKSPACE_PRIVILEGES"

Prerequisites

In order for Chronicle to ingest Workspace ChromeOS devices, complete the following steps:

  1. Enable the Admin SDK API on your Google Cloud project.
  2. Create a Service Account which will be used to authenticate against the Admin API.
  3. Generate a JSON key for the Service Account.
  4. Create a domain-wide delegation for the Service Account with the following OAuth scope:
    • https://www.googleapis.com/auth/admin.directory.rolemanagement.readonly
  5. Create a Workspace user and assign it a super admin role.
  6. Locate your Workspace customer ID.

Type-specific request fields

Field Required Description
details.workspacePrivilegesSettings.authentication.tokenEndpoint Yes The value of the token_uri field in the JSON key for the service account created to access the admin API.
details.workspacePrivilegesSettings.authentication.claims.issuer Yes The value of the client_email field in the JSON key for the service account created to access the admin API.
details.workspacePrivilegesSettings.authentication.claims.subject Yes The email address of the Workspace admin user.
details.workspacePrivilegesSettings.authentication.claims.audience Yes The value of the token_uri field in the JSON key for the service account created to access the admin API.
details.workspacePrivilegesSettings.authentication.rsCredentials.privateKey Yes The value of the private_key field in the JSON key for the service account created to access the admin API. Note that literal newline characters (\n) should be replaced with carriage returns. Note that the field name is "rsCredentials", and not "rsaCredentials".
details.workspacePrivilegesSettings.workspaceCustomerId Yes The Workspace customer ID. Note that the customer ID must have a leading 'C' character. The customer ID may appear differently depending on where in the Google admin console it is found. If the customer ID you have does not have a leading 'C' then prepend what you have with a 'C'.

Sample CreateFeed request

{
 "details": {
   "feedSourceType": "API",
   "logType": "WORKSPACE_PRIVILEGES",
   "workspacePrivilegesSettings": {
     "authentication": {
       "tokenEndpoint": "https://accounts.google.com/o/oauth2/token",
       "claims": {
         "issuer": "service-account@project.iam.gserviceaccount.com",
         "subject": "user@domain.com",
         "audience": "https://accounts.google.com/o/oauth2/token"
       },
       "rsCredentials": {
         "privateKey": "-----BEGIN PRIVATE KEY-----
MIIEvQIBADANBgkqhkiG9w0BAQE...o/gyVcgdkPBHC+sLG8g702Q=
         -----END PRIVATE KEY-----"
       },
     },
     "workspaceCustomerId": "C1e2x3ample",
   }
 }
}

Workspace Users

Data source Ingest schedule details.feedSourceType details.logType
admin.googleapis.com Every 24 hours "API" "WORKSPACE_USERS"

Prerequisites

For Chronicle to ingest Workspace ChromeOS devices, complete the following steps:

  1. Enable the Admin SDK API on your Google Cloud project.
  2. Create a Service Account which will be used to authenticate against the Admin API.
  3. Generate a JSON key for the Service Account.
  4. Create a domain-wide delegation for the Service Account with the following OAuth scope:
    • https://www.googleapis.com/auth/admin.directory.user.readonly
  5. Create a Workspace user and assign it an admin role which includes Admin API User read privileges, or create a custom role which includes that privilege.
  6. Locate your Workspace customer ID.

Type-specific request fields

Field Required Description
details.workspaceUserSettings.authentication.tokenEndpoint Yes The value of the token_uri field in the JSON key for the service account created to access the admin API.
details.workspaceUserSettings.authentication.claims.issuer Yes The value of the client_email field in the JSON key for the service account created to access the admin API.
details.workspaceUserSettings.authentication.claims.subject Yes The email address of the Workspace admin user with the Admin API User read privilege.
details.workspaceUserSettings.authentication.claims.audience Yes The value of the token_uri field in the JSON key for the service account created to access the admin API.
details.workspaceUserSettings.authentication.rsCredentials.privateKey Yes The value of the private_key field in the JSON key for the service account created to access the admin API. Note that literal newline characters (\n) should be replaced with carriage returns. Note that the field name is "rsCredentials", and not "rsaCredentials".
details.workspaceUserSettings.workspaceCustomerId Yes The Workspace customer ID. Note that the customer ID must have a leading 'C' character. The customer ID may appear differently depending on where in the Google admin console it is found. If the customer ID you have does not have a leading 'C' then prepend what you have with a 'C'.

Sample CreateFeed request

{
 "details": {
   "feedSourceType": "API",
   "logType": "WORKSPACE_USERS",
   "workspaceUserSettings": {
     "authentication": {
       "tokenEndpoint": "https://accounts.google.com/o/oauth2/token",
       "claims": {
         "issuer": "service-account@project.iam.gserviceaccount.com",
         "subject": "user@domain.com",
         "audience": "https://accounts.google.com/o/oauth2/token"
       },
       "rsCredentials": {
         "privateKey": "-----BEGIN PRIVATE KEY-----
MIIEvQIBADANBgkqhkiG9w0BAQE...o/gyVcgdkPBHC+sLG8g702Q=
         -----END PRIVATE KEY-----"
       },
     },
     "workspaceCustomerId": "C1e2x3ample",
   }
 }
}

Source type

When specifying a source URI, you must also indicate the type of file indicated by the URI and what specifically you wish to transfer.

details.sourceType Source type
FILES The URI points to a single file which will be ingested with each execution of the feed.
FOLDERS The URI points to a directory. All files contained within the directory will be ingested with each execution of the feed.
FOLDERS_RECURSIVE The URI points to a directory. All files and directories contains within the indicated directory will be ingested, including all files and directories within those directories, and so on.

Source deletion option

For certain cloud-based object stores, it is possible for Chronicle to delete the source files after they have been successfully transferred to Chronicle. This is one way to save on storage costs should the those logs exist in that location only for ingestion by Chronicle.

The possible values for sourceDeletionOption are as follows:

  • SOURCE_DELETION_NEVER: Never delete files from the source.
  • SOURCE_DELETION_ON_SUCCESS: Delete files and empty directories from the source after successful ingestion.
  • SOURCE_DELETION_ON_SUCCESS_FILES_ONLY: Delete files from the source after successful ingestion.

Feed Schema

The Feed Management API provides a way to help clients construct valid requests for creating or updating feeds. This is done by something called a "feed schema". The feed schema is a machine readable static collection of information about how various CreateFeed and UpdateFeed request fields are compatible with one another, as well as additional human-readable information about each field.

Specifically, the feed schema contains:

  • Information about each valid feed source type:
    • A human-readable name
    • A human-readable description
    • Whether feeds with a given feed source type can be modified using the API, or are read-only
  • Information about each log type:
    • A human-readable name
    • Whether feeds with a given log type can be modified using the API, or are read-only
  • Which log types are compatible with which feed source types
  • Information about the specific fields to specify for each valid combination of log type and feed source type:
    • A human-readable field name and description
    • Compatibility with other fields
    • Semantic type (e.g. URI, "secret", etc)
    • Whether the field is required
    • What a valid value for the field looks like

The schema can be accessed using a few different methods.

GetFeedSchema

This method returns a structure representing the entire feed schema. The schema is comprised of a list of "feed source type schemas" each of which describe the supported feed source types. Each feed source types' schema contains a list of "log type schemas" that correspond to the set of log types that are compatible with the feed source type and describe the log type. Each log type schema contains a list of "details field schemas" that describe those fields you would set when issuing a CreateFeed request, for instance, or expect to see as a result of a ListFeed or GetFeed response. The field schemas specified are unique to the particular combination of log type and feed source type.

Request

https://backstory.googleapis.com/v1/feedSchema

Response

{
  "feedSourceTypeSchemas": [{
      "name": "feedSourceTypeSchemas/AMAZON_S3",
      "displayName": "Amazon S3",
      "description": "Amazon Simple Storage Service, a service offered by Amazon Web Services that provides object storage through a web service interface",
      "feedSourceType": "AMAZON_S3",
      "logTypeSchemas": [{
          "name": "feedSourceTypeSchemas/AMAZON_S3/logTypeSchemas/AWS_CLOUDTRAIL",
          "displayName": "AWS Cloudtrail",
          "logType": "AWS_CLOUDTRAIL",
          "detailsFieldSchemas": [{
              "fieldPath": "details.amazon_s3_settings.authentication.access_key_id",
              "displayName": "Access key ID",
              "description": "An account access key that is a 20-character alphanumeric string, for example AKIAIOSFODNN7EXAMPLE",
              "type": "STRING",
              "exampleInput": "AKIAIOSFODNN7EXAMPLE",
            },
            ...
            {
              "fieldPath": "details.amazon_s3_settings.s3_uri",
              "displayName": "S3 URI",
              "description": "The S3 bucket source URI",
              "type": "STRING_URI",
              "isRequired": true,
              "exampleInput": "s3://cs-prod-cannon-00afe0c847a8/data/",
            }],
        },
        ...
        {
          "name": "feedSourceTypeSchemas/AMAZON_S3/logTypeSchemas/ABNORMAL_SECURITY",
          "displayName": "Abnormal Security",
          "logType": "ABNORMAL_SECURITY",
          ...
        }],
    },
    ...
    {
      "name": "feedSourceTypeSchemas/AMAZON_SQS",
      "displayName": "Amazon SQS",
      "description": "Amazon Simple Queue Service, a service offered by Amazon Web Services that provides fully managed message queuing service to transfer messages asynchronously",
      "feedSourceType": "AMAZON_SQS",
      ...
    }],
}

ListFeedSourceTypeSchemas

This method returns information about all feed source types.

Request
https://backstory.googleapis.com/v1/feedSourceTypeSchemas
Sample Response
{
  "feedSourceTypeSchemas": [{
      "name": "feedSourceTypeSchemas/AMAZON_S3",
      "displayName": "Amazon S3",
      "description": "Amazon Simple Storage Service, a service offered by Amazon Web Services that provides object storage through a web service interface",
      "feedSourceType": "AMAZON_S3",
    },
    ...
    {
      "name": "feedSourceTypeSchemas/AMAZON_SQS",
      "displayName": "Amazon SQS",
      "description": "Amazon Simple Queue Service, a service offered by Amazon Web Services that provides fully managed message queuing service to transfer messages asynchronously",
      "feedSourceType": "AMAZON_SQS",
    }],
}

ListLogTypeSchemas

This method returns information about all log types compatible with a particular feed source type.

Request
https://backstory.googleapis.com/v1/feedSourceTypeSchemas/{feed source type}/logTypeSchemas
Sample Request
https://backstory.googleapis.com/v1/feedSourceTypeSchemas/AMAZON_S3/logTypeSchemas
Sample Response
{
  "logTypeSchemas": [{
      "name": "feedSourceTypeSchemas/AMAZON_S3/logTypeSchemas/AWS_CLOUDTRAIL",
      "displayName": "AWS Cloudtrail",
      "logType": "AWS_CLOUDTRAIL",
    },
    ...
    {
      "name": "feedSourceTypeSchemas/AMAZON_S3/logTypeSchemas/ABNORMAL_SECURITY",
      "displayName": "Abnormal Security",
      "logType": "ABNORMAL_SECURITY",
      ...
    }],
}

GetLogTypeSchema

This method returns detailed information about all the fields necessary to configure a feed for a particular source type and log type.

Request
https://backstory.googleapis.com/v1/feedSourceTypeSchemas/{feed source type}/logTypeSchemas/{log type}
Sample Request
https://backstory.googleapis.com/v1/feedSourceTypeSchemas/AMAZON_S3/logTypeSchemas/AWS_CLOUDTRAIL
Sample Response
{
  "name": "feedSourceTypeSchemas/AMAZON_S3/logTypeSchemas/AWS_CLOUDTRAIL",
  "displayName": "AWS Cloudtrail",
  "logType": "AWS_CLOUDTRAIL",
  "detailsFieldSchemas": [{
      "fieldPath": "details.amazon_s3_settings.authentication.access_key_id",
      "displayName": "Access key ID",
      "description": "An account access key that is a 20-character alphanumeric string, for example AKIAIOSFODNN7EXAMPLE",
      "type": "STRING",
      "exampleInput": "AKIAIOSFODNN7EXAMPLE",
    },
    ...
    {
      "fieldPath": "details.amazon_s3_settings.s3_uri",
      "displayName": "S3 URI",
      "description": "The S3 bucket source URI",
      "type": "STRING_URI",
      "isRequired": true,
      "exampleInput": "s3://cs-prod-cannon-00afe0c847a8/data/",
    }],
}