Google Security Operations CLI User Guide

Supported in:

As an alternative to using the Google Security Operations graphical user interface, advanced users can use the Google Security Operations command-line interface (CLI), chronicle_cli, to initiate Google Security Operations workflows.

You can use the Google Security Operations CLI for these workflows:

Google Security Operations CLI commands use the following syntax (COMMAND and ARGUMENT are required, but OPTIONS is optional):

$ chronicle_cli COMMAND ARGUMENT [OPTIONS]

For example, to create a new feed using the feed management workflows, use the following command:

$ chronicle_cli feeds create

Before you begin

Before installing Google Security Operations CLI, do the following:

Installation

This section provides information on how to install the Google Security Operations CLI in your environment.

  1. Create and activate the virtual environment venv.

  2. Clone the repository using following command:

    git clone https://github.com/chronicle/cli.git
    
  3. Open a terminal and install all the required dependent packages in your virtual environment by executing the following command:

    $ cd cli
    $ (env) pip install -r requirements.txt
    
  4. Install the Google Security Operations binary by executing the following command:

    $ (env) python3 -m pip install --editable .
    
  5. Verify that the installation is successful by executing the following command:

    $ chronicle_cli --help
    
Sample output
Usage: chronicle_cli [OPTIONS] COMMAND [ARGS]...

  Google Security Operations CLI is a CLI tool for managing Google Security Operations user workflows for e.g.
  Feed Management workflows.

Options:
  -h, --help  Show this message and exit.

Commands:
  feeds  Feed Management Workflows

Options

You can override the default configuration by providing additional flags when you run commands.

Set a default region (--region)

You can select a region by passing the --region flag with the command and the API calls will be made to the appropriate Google Security Operations region backend.

You can set the following regions:

  • ASIA-NORTHEAST1
  • ASIA-SOUTH1
  • ASIA-SOUTHEAST1
  • AUSTRALIA-SOUTHEAST1
  • EUROPE
  • EUROPE-WEST2
  • EUROPE-WEST3
  • EUROPE-WEST6
  • EUROPE-WEST12
  • ME-CENTRAL1
  • ME-CENTRAL2
  • ME-WEST1
  • NORTHAMERICA-NORTHEAST2
  • US

If you don't specify the region, the default region is set to US.

Feed management workflows

You can use the Google Security Operations CLI to create and manage data feeds to your Google SecOps instance.

Commands

feeds command

The feeds command takes these arguments:

  • create
  • update
  • get
  • list
  • delete
  • enable
  • disable

Usage syntax:

$ chronicle_cli feeds ARGUMENT [OPTIONS]

Arguments

create argument

Creates a new feed.

Sample usage
$ chronicle_cli feeds create --help
Usage: chronicle_cli feeds create [OPTIONS]

  Create a feed

Options:
  --url TEXT                      Base URL to be used for API calls.
  --region 
                                  Select region.
  --verbose                       Prints verbose output to the console.
  -c, --credential-path TEXT      Path of Service Account JSON. Default: /usr/
                                  local/google/home/<user>/.chronicle_cli/chronicle_credentials.json
  -h, --help                      Show this message and exit.
Sample output
  • For Windows platform
====================================
========== Set Properties ==========
====================================

List of Source types:
1. Amazon S3
2. Amazon SQS
3. Google Cloud Storage
4. HTTP(S) URI
5. Microsoft Azure Blob Storage
6. SFTP
7. Third party API

[Source type] Enter your choice: 7

You have selected Third party API

List of Log types:

(i) How to select log type?
  - Press ENTER key (scrolls one line at a time) or SPACEBAR key (display next screen).
  - Note down the choice number for the log type that you want to select.
  - Press 'q' to quit and enter that choice number.
=============================================================================
1. Anomali
2. Azure AD
3. Azure AD Directory Audit
4. Azure AD Organizational Context
5. Cloud Passage
6. Duo Auth
7. Duo User Context
8. Fox-IT
9. Imperva
10. Microsoft Graph API Alerts:

[Log type] Enter your choice: 7

You have selected Duo User Context

Enter feed display name: my_duo_user_context_feed


======================================
=========== Input Parameters =========
======================================
(*) - Required fields.
Password/secret inputs are hidden.

(*) Username (Username to authenticate as)
=> USERNAME

(*) Secret (Secret to authenticate with)
=>

(*) API hostname (The fully qualified domain name for your instance of the API, having the form "api-xxxxxxxx.duosecurity.com")
=> api-xxxxxxxx.duosecurity.com

Feed created successfully with Feed ID: 9cfce415-97df-413b-8e38-e7c747f9ed38
  • For other platforms such as Linux/Ubuntu/CentOS/MacOS
====================================
========== Set Properties ==========
====================================

List of Source types:
1. Amazon S3
2. Amazon SQS
3. Google Cloud Storage
4. HTTP(S) URI
5. Microsoft Azure Blob Storage
6. SFTP
7. Third party API

[Source type] Enter your choice: 7

You have selected Third party API

List of Log types:

(i) How to select log type?
  - Press Up/b or Down/z keys to paginate.
  - To switch case-sensitivity, press '-i' and press enter. By default, search
    is case-sensitive.
  - To search for specific log type, press '/' key, enter text and press enter.
  - Note down the choice number for the log type that you want to select.
  - Press 'q' to quit and enter that choice number.
  - Press `h` for all the available options to navigate the list.
=============================================================================
1. Anomali
2. Azure AD
3. Azure AD Directory Audit
4. Azure AD Organizational Context
5. Cloud Passage
6. Duo Auth
7. Duo User Context
8. Fox-IT
9. Imperva:

[Log type] Enter your choice: 7

You have selected Duo User Context
======================================
======================================
(*) - Required fields.
Password/secret inputs are hidden.

(*) Username (Username to authenticate as)
=> USERNAME

(*) Secret (Secret to authenticate with)
=>

(*) API hostname (The fully qualified domain name for your instance of the API, having the form "api-xxxxxxxx.duosecurity.com")
=> api-xxxxxxxx.duosecurity.com

Feed created successfully with Feed ID: 9cfce415-97df-413b-8e38-e7c747f9ed38

If feed creation fails, the next time you are asked to retry the failed feed creation. You can opt to retry or to continue creating a new feed. The retry mechanism allows you to interactively change values provided in the previous failed attempt. Press Enter to reuse the same value for an option in the feed creation flow.

Sample output
====================================
========== Set Properties ==========
====================================

List of Source types:
1. Amazon S3
2. Amazon SQS
3. Google Cloud Storage
4. HTTP(S) URI
5. Microsoft Azure Blob Storage
6. SFTP
7. Third party API

[Source type] Enter your choice: 7

You have selected Third party API

[Log type] Enter your choice: 6

You have selected Duo Auth

Enter feed display name: my_duo_auth_feed


======================================
=========== Input Parameters =========
======================================
(*) - Required fields.
Password/secret inputs are hidden.

(*) Username (Username to authenticate as)
=> test

(*) Secret (Secret to authenticate with)
=>

(*) API hostname (The fully qualified domain name for your instance of the API, having the form "api-xxxxxxxx.duosecurity.com")
=> test.com

Error occurred while creating feed.
Response Code: 400.
Error: generic::invalid_argument: failed to create feed for the customer (ID: ed19f037-2354-43df-bfbf-350362b45844): failed to create feed for the customer (ID: ed19f037-2354-43df-bfbf-350362b45844): failed to create feed because of the following errors in the request: generic::invalid_argument: for Duo feeds, 'hostname' must be specified as "api-xxxxxxxx.duosecurity.com", e.g. "api-eval.duosecurity.com"

$ chronicle_cli feeds create

Looks like there was a failed feed create/update attempt with source type: Third party API and log type: Duo Auth.
Would you like to retry?

======================================
=========== Input Parameters =========
======================================
(*) - Required fields.
Password/secret inputs are hidden.

(*) Username (Username to authenticate as) [test]
=>

(*) Secret (Secret to authenticate with)
=>

(*) API hostname (The fully qualified domain name for your instance of the API, having the form "api-xxxxxxxx.duosecurity.com") [test.com]
=> api-xxxxxxxx.duosecurity.com

Feed created successfully with Feed ID: 29a2f967-6f6e-4521-bebf-6fb6c7383df6

get argument

Gets details of an existing feed.

Sample usage
$ chronicle_cli feeds get --help
Usage: main feeds get [OPTIONS]

  Get feed details using Feed ID

Options:
  --url TEXT                      Base URL to be used for API calls.
  --region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|EUROPE-WEST12|ME-CENTRAL1|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
                                  Select region.
  --verbose                       Prints verbose output to the console.
  -c, --credential-path TEXT      Path of Service Account JSON. Default: /usr/
                                  local/google/home/<user>/.chronicle_cli/chronicle_credentials.json
  -h, --help                      Show this message and exit.

You must enter the feed ID interactively to get the feed details.

Sample output
Enter Feed ID: 72d9b843-b387-4b17-ab2d-a8497313c89c

Feed Details:
  ID: 72d9b843-b387-4b17-ab2d-a8497313c89c
  Display Name: my_duo_auth_feed
  Source type: Third party API
  Log type: Salesforce
  State: ACTIVE
  Feed Settings:
    API Hostname: myinstance.salesforce.com

list argument

Lists all feeds. This command is used to fetch the details of all the feeds.

Sample usage
$ chronicle_cli feeds list --help
Usage: chronicle_cli feeds list [OPTIONS]

List all feeds

Options:
  --url TEXT                      Base URL to be used for API calls.
  --region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|EUROPE-WEST12|ME-CENTRAL1|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
                                  Select region.
  --verbose                       Prints verbose output to the console.
  --export TEXT                   Export output to specified file path.
  --file-format [TXT|CSV|JSON]    Format of the file to be exported.
  -c, --credential-path TEXT      Path of Service Account JSON. Default: /usr/
                                  local/google/home/<user>/.chronicle_cli/chronicle_credentials.json
  -h, --help                      Show this message and exit.
Sample output
Feed Details:
  ID: 29259301-156b-4b60-ae91-855d15c39f6a
  Source type: Third party API
  Log type: Anomali
  State: INACTIVE
============================================================

Feed Details:
  ID: 292b7629-0250-476c-9fb2-4c8a738ce42c
  Display Name: my_duo_auth_feed
  Source type: Third party API
  Log type: Duo Auth
  State: ACTIVE
  Feed Settings:
    API hostname: api-test.duosecurity.com

============================================================

Feed Details:
  ID: 0d063a7f-34a1-4dd0-9dcf-9c7a0bb03e65
  Source type: Third party API
  Log type: Workspace Activities
  State: ACTIVE
  Feed Settings:
    Customer ID: C12abc
    Applications: ['drive', 'login']

============================================================

If you want to export the data, then you can specify the absolute/relative path of the file to be exported along with the format of the file (CSV/TXT/JSON). The default file format is CSV.

Sample output
Feed Details:
  ID: 29259301-156b-4b60-ae91-855d15c39f6a
  Source type: Third party API
  Log type: Anomali
  State: INACTIVE
============================================================

Feed Details:
  ID: 292b7629-0250-476c-9fb2-4c8a738ce42c
  Display Name: my_duo_auth_feed
  Source type: Third party API
  Log type: Duo Auth
  State: ACTIVE
  Feed Settings:
    API hostname: api-test.duosecurity.com

============================================================

Feed Details:
  ID: 0d063a7f-34a1-4dd0-9dcf-9c7a0bb03e65
  Source type: Third party API
  Log type: Workspace Activities
  State: ACTIVE
  Feed Settings:
    Customer ID: C12abc
    Applications: ['drive', 'login']

============================================================
Feed list details exported successfully to: /usr/local/google/home/<user>/out/chronicle-cli/output.txt

update argument

Updates an existing feed.

Sample usage
$ chronicle_cli feeds update
Usage: chronicle_cli feeds update [OPTIONS]

  Update feed details using Feed ID

Options:
  --url TEXT                      Base URL to be used for API calls.
  --region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|EUROPE-WEST12|ME-CENTRAL1|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
                                  Select region.
  --verbose                       Prints verbose output to the console.
  -c, --credential-path TEXT      Path of Service Account JSON. Default: /usr/
                                  local/google/home/<user>/.chronicle_cli/chronicle_credentials.json
  -h, --help                      Show this message and exit.

After running the command, enter a feed ID and all the field values again. Press Enter to reuse the old value.

Sample output
Enter Feed ID: ea28d66b-d81b-4b4d-ae16-3b1cd98132ca
Press Enter if you don't want to update.

Enter feed display name[old_display_name]:

(*) Username (Username to authenticate as)
=> USERNAME

(*) Secret (Secret to authenticate with)
=>

(*) API hostname (The fully qualified domain name for your instance of the API, having the form "api-xxxxxxxx.duosecurity.com") [api-xxxxxxxx.duosecurity.com]
=>

Feed updated successfully with Feed ID: ea28d66b-d81b-4b4d-ae16-3b1cd98132ca
Enter Feed ID: 29a2f967-6f6e-4521-bebf-6fb6c7383df6
Press Enter if you don't want to update.

Enter feed display name[]: my_feed_display_name

(*) Username (Username to authenticate as)
=> test1

(*) Secret (Secret to authenticate with)
=>

(*) API hostname (The fully qualified domain name for your instance of the API, having the form "api-xxxxxxxx.duosecurity.com") [api-xxxxxxxx.duosecurity.com]
=> test.com

Error occurred while updating feed. Response code: 400.
Error: generic::invalid_argument: failed to update feed for the customer (ID: ed19f037-2354-43df-bfbf-350362b45844): failed to edit feed because of the following errors in the request: generic::invalid_argument: for Duo feeds, 'hostname' must be specified as "api-xxxxxxxx.duosecurity.com", e.g. "api-eval.duosecurity.com"

$ chronicle_cli feeds update
Enter Feed ID: 29a2f967-6f6e-4521-bebf-6fb6c7383df6

Looks like there was a failed feed create/update attempt with source type: Third party API and log type: Duo Auth.
Would you like to retry?

Press Enter if you don't want to update.

(*) Username (Username to authenticate as) [test1]
=>

(*) Secret (Secret to authenticate with)
=>

(*) API hostname (The fully qualified domain name for your instance of the API, having the form "api-xxxxxxxx.duosecurity.com") [test.com]
=> api-devtest.duosecurity.com

Feed updated successfully with Feed ID: 29a2f967-6f6e-4521-bebf-6fb6c7383df6

If the feed update fails and you enter the same feed ID, then a prompt is displayed to retry the failed feed or to restart the process. If you enter a feed ID which does not match the failed feed ID, then the option to retry is not displayed and the normal process of updating the feed continues. The retry mechanism allows you to change option values provided in the previous failed attempt in an interactive way. Press Enter to reuse the same value for an option in the feed update flow.

Sample output
Enter Feed ID: 51574667-dee6-408b-a5fc-0e07d3e9a429

Looks like there was a failed feed create/update attempt with source type: Third party API and log type: Duo Auth.
Would you like to retry?

Press Enter if you don't want to update.

Enter feed display name[old_display_name]:

(*) Username (Username to authenticate as) [TEEST]
=> TEST

(*) Secret (Secret to authenticate with)
=>

(*) API hostname (The fully qualified domain name for your instance of the API, having the form "api-xxxxxxxx.duosecurity.com") [asd]
=> api-xxxxxxxx.duosecurity.com

Feed updated successfully with Feed ID: 51574667-dee6-408b-a5fc-0e07d3e9a429

delete argument

Use this argument to delete a feed using a feed ID. On execution, it asks for the ID of the feed to be deleted.

Sample usage
$ chronicle_cli feeds delete --help
Usage: chronicle_cli feeds delete [OPTIONS]

  Delete a feed

Options:
  --url TEXT                      Base URL to be used for API calls.
  --region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|EUROPE-WEST12|ME-CENTRAL1|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
                                  Select region.
  -c, --credential-path TEXT      Path of Service Account JSON. Default: /usr/
                                  local/google/home/<user>/.chronicle_cli/chronicle_credentials.json
  -h, --help                      Show this message and exit.
Sample output
Enter Feed ID: b0798c54-ed84-44e7-96d5-cbe208f28e49

Feed (ID: b0798c54-ed84-44e7-96d5-cbe208f28e49) deleted successfully.

enable argument

Enables a feed.

Sample usage
$ chronicle_cli feeds enable --help
Usage: main feeds enable [OPTIONS]

  Enable feed for the given Feed ID

Options:
  --url TEXT                      Base URL to be used for API calls.
  --region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|EUROPE-WEST12|ME-CENTRAL1|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
                                  Select region.
  --verbose                       Prints verbose output to the console.
  -c, --credential-path TEXT      Path of Service Account JSON. Default: /usr/
                                  local/google/home/<user>/.chronicle_cli/chro
                                  nicle_credentials.json
  -h, --help                      Show this message and exit.

You must enter the feed ID to enable the feed.

Sample output
Enter Feed ID: 29259301-156b-4b60-ae91-855d15c39f6a
Feed with ID: 29259301-156b-4b60-ae91-855d15c39f6a enabled successfully.

disable argument

Disables a feed.

Sample usage
$ chronicle_cli feeds disable --help
Usage: main feeds disable [OPTIONS]

  Disable feed for the given Feed ID

Options:
  --url TEXT                      Base URL to be used for API calls.
  --region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|EUROPE-WEST12|ME-CENTRAL1|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
                                  Select region.
  --verbose                       Prints verbose output to the console.
  -c, --credential-path TEXT      Path of Service Account JSON. Default: /usr/
                                  local/google/home/<user>/.chronicle_cli/chro
                                  nicle_credentials.json
  -h, --help                      Show this message and exit.

You must enter the feed ID to disable the feed.

Sample output
Enter Feed ID: 29259301-156b-4b60-ae91-855d15c39f6a
Feed with ID: 29259301-156b-4b60-ae91-855d15c39f6a disabled successfully.

Options

Help (-h / --help)

Use -h or --help option to view usage/description for any command/option.

Sample usage
$ chronicle_cli feeds get -h
Usage: main feeds get [OPTIONS]

  Get feed details using Feed ID

Options:
  --url TEXT                      Base URL to be used for API calls.
  --region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|EUROPE-WEST12|ME-CENTRAL1|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
                                  Select region.
  --verbose                       Prints verbose output to the console
  -c, --credential-path TEXT      Path of Service Account JSON. Default: /usr/
                                  local/google/home/<user>/.chronicle_cli/chron
                                  icle_credentials.json
  -h, --help                      Show this message and exit.

Credentials Path (-c or --credential-path)

This option allows to specify the path for the service account credentials to be used for the authentication. If this option is not specified, then the Google Security Operations CLI looks for credentials in the default path, i.e. ~/.chronicle_cli (inside a hidden directory named .chronicle_cli under the home directory).

Sample usage
$ chronicle_cli feeds list --credential-path=C:\chronicle_credentials.json

Verbose (--verbose)

The Google Security Operations CLI prints more details to the console, such as HTTP requests and responses, when this flag is used.

Sample usage
$ chronicle_cli feeds list --verbose

Export (--export)

This option allows you to specify the file path where the output of the list command will be exported. Both relative and absolute paths are supported.

Sample usage
$ chronicle_cli feeds list --export=$HOME/listFeedsResponse.txt

File Format (--file-format)

This option allows you to specify the file format of the contents exported with the list command. There are three formats supported: CSV, JSON and TXT. If this option is not specified with the --export option, then CSV format would be used as default.

Sample usage
$ chronicle_cli feeds list --export=$HOME/listFeedsResponse.txt --file-format=TXT
Sample output
CSV format
ID,Display Name,Source type,Log type,State,Feed Settings
29259301-156b-4b60-ae91-855d15c39f6a,,Third party API,Anomali,INACTIVE,
292b7629-0250-476c-9fb2-4c8a738ce42c,my_duo_auth_feed,Third party API,Duo Auth,ACTIVE,API hostname: api-xxxxxxxxabjdsfklsadlfnsafs.duosecurity.com
0d063a7f-34a1-4dd0-9dcf-9c7a0bb03e65,,Third party API,Workspace Activities,ACTIVE,"Customer ID: C12abc    Applications: ['drive', 'login']"
TXT format
Feed Details:
  ID: 29259301-156b-4b60-ae91-855d15c39f6a
  Source type: Third party API
  Log type: Anomali
  State: INACTIVE
============================================================

Feed Details:
  ID: 292b7629-0250-476c-9fb2-4c8a738ce42c
  Display Name: my_duo_auth_feed
  Source type: Third party API
  Log type: Duo Auth
  State: ACTIVE
  Feed Settings:
    API hostname: api-test.duosecurity.com

============================================================
JSON format
[
  {
    "name": "feeds/29259301-156b-4b60-ae91-855d15c39f6a",
    "details": {
      "logType": "ANOMALI_IOC",
      "feedSourceType": "API",
      "anomaliSettings": {}
    },
    "feedState": "INACTIVE"
  },
  {
    "name": "feeds/292b7629-0250-476c-9fb2-4c8a738ce42c",
    "details": {
      "logType": "DUO_AUTH",
      "feedSourceType": "API",
      "duoAuthSettings": {
        "hostname": "api-test.duosecurity.com"
      }
    },
    "feedState": "ACTIVE",
    "displayName": "my_duo_auth_feed"
  }
]

Region (--region)

You can select the region by passing the --region flag when you run a command. For more information on how to set the default region, see Set default region.

Troubleshooting

This section shows the output displayed on the console against different types of response codes received from API response.

get argument response codes

Response Code Console Output
404 Invalid feed ID. Please enter valid feed ID.
400 Feed does not exist.
Any other response code Error while fetching feed. Response Code: {status code} Error: {error message}

list argument response codes

Response Code Console Output
Any response code other than 200 Error while fetching list of feeds. Response Code: {status code} Error: {error message}
Any feed in the list failed to fetch At the end of the console output a list will be printed with the details of the feed ID and corresponding error message.

create argument response codes

Response Code Console Output
Any response code other than 200 Error occurred while creating feed. Response Code: {status code} Error: {error message}

update argument response codes

Response Code Console Output
Any response code other than 200 Error occurred while updating feed. Response Code: {status code} Error: {error message}

delete argument response codes

Response Code Console Output
404 Invalid feed ID. Please enter valid feed ID.
400 Feed does not exist.
Any other response code Error while deleting feed. Response Code: {status code} Error: {error message}

enable argument response codes

Response Code Console Output
404 Invalid feed ID. Please enter valid feed ID.
400 Feed does not exist.
Any other response code Error while enabling feed. Response Code: {status code} Error: {error message}

disable argument response codes

Response Code Console Output
404 Invalid feed ID. Please enter valid feed ID.
400 Feed does not exist.
Any other response code Error while disabling feed. Response Code: {status code} Error: {error message}

Other errors or exceptions

Exception Console Output
KeyError Failed to find key {key name} in the response.
Exception Failed with exception: {exception details}
Credential file missing Failed with exception: [Errno 2] No such file or directory: '/usr/local/google/home//.chronicle_cli/chronicle_credentials.json'

You need to place credentials in the expected directory. Refer to Installation.

Parser management v2 user workflows

You can use Google SecOps CLI to manage configuration-based normalizer (CBN) parsers. We recommend using parser management v2 CLI commands.

Commands

parsers command

The parsers command takes these arguments:

  • list_parsers
  • list_extensions
  • run_parser
  • submit_parser
  • submit_extension
  • delete_parser
  • delete_extension
  • deactivate_parser
  • activate_parser
  • get_parser
  • get_extension
  • get_validation_report

Usage syntax:

$ chronicle_cli parsers ARGUMENT [OPTIONS]

Arguments

All CBN parser management workflows in the Google SecOps CLI are interactive. If required, you can also use the command options.

list_parsers argument

Lists all parsers.

$ chronicle_cli parsers list_parsers -h
Usage: chronicle_cli parsers list_parsers [OPTIONS] PROJECT_ID CUSTOMER_ID LOG_TYPE

  [New]List all parsers for a given customer

Options:
  -s, --state [ALL|ACTIVE|INACTIVE]
                                  Filter on Parser State.
  -f, --file-format [TXT|JSON]    Format of the file to be exported.
  --export TEXT                   Export output to specified file path.
  --env [prod|test]               Optionally specify the environment for API
                                  calls.
  --region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|EUROPE-WEST12|ME-CENTRAL1|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
                                  Select region.
  --verbose                       Prints verbose output to the console.
  -c, --credential_file TEXT      Path of Service Account JSON. Default: /usr/
                                  local/home/<user>/.chronicle_cli
                                  /chronicle_credentials.json
  --v2                            Enable v2 commands.
  -h, --help                      Show this message and exit.
Sample output
Fetching list of parsers...

Parser Details:
  Parser ID: 1242538299340357633
  Log type: GCP_CLOUDAUDIT
  State: INACTIVE
  Type: CUSTOM
  Author: -
  Validation Report ID: 44684d8a-1d01-4e69-ab50-2e2d6e3ef3b2
  Create Time: 2023-07-05T05:36:31.121236Z

============================================================

Parser Details:
  Parser ID: 3840440184193679361
  Log type: GCP_CLOUDAUDIT
  State: INACTIVE
  Type: CUSTOM
  Author: -
  Validation Report ID: 3d2e1bdb-2793-48d1-a485-4f4748095cb8
  Create Time: 2023-04-14T09:15:13.718842Z

============================================================

Parser Details:
  Parser ID: 3651720008402206721
  Log type: GCP_SECURITYCENTER_ERROR
  State: ACTIVE
  Type: CUSTOM
  Author: -
  Validation Report ID: -
  Create Time: 2023-03-30T09:54:20.414510Z

============================================================

list_extensions argument

Lists all parser extensions.

$ chronicle_cli parsers list_extensions -h
Usage: chronicle_cli parsers list_extensions [OPTIONS] PROJECT_ID CUSTOMER_ID LOG_TYPE

  [New]List all extensions for a given customer

Options:
  -f, --file-format [TXT|JSON]    Format of the file to be exported.
  --export TEXT                   Export output to specified file path.
  --env [prod|test]               Optionally specify the environment for API
                                  calls.
  --region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|EUROPE-WEST12|ME-CENTRAL1|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
                                  Select region.
  --verbose                       Prints verbose output to the console.
  -c, --credential_file TEXT      Path of Service Account JSON. Default: /usr/
                                  local/home/<user>/.chronicle_cli
                                  /chronicle_credentials.json
  --v2                            Enable v2 commands.
  -h, --help                      Show this message and exit.
Sample output
Fetching list of Parser Extensions...

ParserExtension Details:
  ParserExtension ID: 7b948bfb-d3f5-4922-9153-a20e75085990
  Log type: BRO_DNS
  State: VALIDATED
  Validation Report ID: 6ef30ad9-db89-4f30-80f3-0f79758ff3c2
  Create Time: 2023-07-06T03:58:26.594863Z
  State Last Changed Time: 2023-07-06T03:58:26.667151Z
  Last Live Time: 2023-07-06T03:58:28.019050Z

============================================================

ParserExtension Details:
  ParserExtension ID: 0fd9129b-d02b-42f7-912a-04b0bba0e0a7
  Log type: GCP_DNS
  State: LIVE
  Validation Report ID: 1965880f-7cd7-4943-9adf-4bff0041793d
  Create Time: 2023-05-12T08:12:17.090559Z
  State Last Changed Time: 2023-05-12T08:12:17.271615Z
  Last Live Time: 2023-05-12T08:12:27.244342Z

============================================================

ParserExtension Details:
  ParserExtension ID: d9df9d75-bb3a-4c28-b18d-69a608762ecc
  Log type: GCP_VPC_FLOW
  State: REJECTED
  Validation Report ID: c59ef2ab-4a70-4373-bdc8-067c39ca5a40
  Create Time: 2023-04-13T04:43:12.884287Z
  State Last Changed Time: 2023-04-13T04:43:13.288338Z
  Last Live Time: -

============================================================

run_parser argument

To validate the parser against given logs, use the following command:

$ chronicle_cli parsers run_parser -h
Usage: chronicle_cli parsers run_parser [OPTIONS] PROJECT_ID CUSTOMER_ID LOG_TYPE
                               PARSER_CONFIG_FILE LOG_FILE

  [New]Run a parser(with extension) against given logs

Options:
  --parserextension_config_file TEXT
                                  Path to extension config file.
  --env [prod|test]               Optionally specify the environment for API
                                  calls.
  --region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|EUROPE-WEST12|ME-CENTRAL1|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
                                  Select region.
  --verbose                       Prints verbose output to the console.
  -c, --credential_file TEXT      Path of Service Account JSON. Default: /usr/
                                  local/home/<user>/.chronicle_cli
                                  /chronicle_credentials.json
  --v2                            Enable v2 commands.
  -h, --help                      Show this message and exit.
Sample output
Running parser(with extension) against given logs...

{"host_ip": "1.1.1.1"}
{'events': [{'event': {'metadata': {'eventTimestamp': '2023-06-26T08:45:10Z', 'eventType': 'GENERIC_EVENT', 'logType': 'BRO_DNS'}, 'principal': {'ip': ['1.1.1.1']}}}]}
some thing
{}

Runtime: 1.2396s

submit_parser argument

Submits a new parser. The submitted parser goes through validation and the existing parser is promoted to a rollback candidate.

$ chronicle_cli parsers submit_parser -h
Usage: main parsers submit_parser [OPTIONS] PROJECT_ID CUSTOMER_ID LOG_TYPE
                                  CONFIG_FILE [AUTHOR]

  [New]Submit a new parser

Options:
  --skip_validation_on_no_logs    Skip validation if no logs are found.
  --env [prod|test]               Optionally specify the environment for API
                                  calls.
  --region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|EUROPE-WEST12|ME-CENTRAL1|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
                                  Select region.
  --verbose                       Prints verbose output to the console.
  -c, --credential_file TEXT      Path of Service Account JSON. Default: /usr/
                                  local/home/<user>/.chronicle_cli
                                  /chronicle_credentials.json
  --v2                            Enable v2 commands.
  -h, --help                      Show this message and exit.
Sample output
Submitting Parser...

Parser Details:
  Parser ID: 12774126091501569
  Log type: GCP_CLOUDAUDIT
  State: INACTIVE
  Type: CUSTOM
  Author: -
  Validation Report ID: -
  Create Time: 2023-07-06T13:58:10.475391Z

============================================================

submit_extension argument

Submits a new parser extension. The submitted parser extension will go through validation. If validation passes, a new parser extension is created.

$ chronicle_cli parsers submit_extension -h
Usage: chronicle_cli parsers submit_extension [OPTIONS] PROJECT_ID CUSTOMER_ID LOG_TYPE
                                     CONFIG_FILE LOG_FILE

  [New]Submit a new extension

Options:
  --env [prod|test]               Optionally specify the environment for API
                                  calls.
  --region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|EUROPE-WEST12|ME-CENTRAL1|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
                                  Select region.
  --verbose                       Prints verbose output to the console.
  -c, --credential_file TEXT      Path of Service Account JSON. Default: /usr/
                                  local/home/<user>/.chronicle_cli
                                  /chronicle_credentials.json
  --v2                            Enable v2 commands.
  -h, --help                      Show this message and exit.
Sample output
Submitting Parser Extension...

ParserExtension Details:
  ParserExtension ID: 88907461-c115-4204-8391-425b7a9cfb2c
  Log type: WORKSPACE_CHROMEOS
  State: NEW
  Validation Report ID: -
  Create Time: 2023-07-06T13:58:10.475391Z
  State Last Changed Time: -
  Last Live Time: -

============================================================

delete_parser argument

Deletes a custom parser. You can start using the prebuilt parser for the given log type.

$ chronicle_cli parsers delete_parser -h
Usage: chronicle_cli parsers delete_parser [OPTIONS] PROJECT_ID CUSTOMER_ID LOG_TYPE
                                  PARSER_ID

  [New]Delete a parser

Options:
  --env [prod|test]               Optionally specify the environment for API
                                  calls.
  --region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|EUROPE-WEST12|ME-CENTRAL1|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
                                  Select region.
  --verbose                       Prints verbose output to the console.
  -c, --credential_file TEXT      Path of Service Account JSON. Default: /usr/
                                  local/home/<user>/.chronicle_cli
                                  /chronicle_credentials.json
  --v2                            Enable v2 commands.
  -h, --help                      Show this message and exit.
Sample output
Deleting Parser…
Parser deleted successfully.

delete_extension argument

Deletes a custom parser. You can start using the prebuilt parser for the given log type.

$ chronicle_cli parsers delete_extension -h
Usage: chronicle_cli parsers delete_extension [OPTIONS] PROJECT_ID CUSTOMER_ID LOG_TYPE
                                     PARSEREXTENSION_ID

  [New]Delete an extension

Options:
  --env [prod|test]               Optionally specify the environment for API
                                  calls.
  --region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|EUROPE-WEST12|ME-CENTRAL1|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
                                  Select region.
  --verbose                       Prints verbose output to the console.
  -c, --credential_file TEXT      Path of Service Account JSON. Default: /usr/
                                  local/home/<user>/.chronicle_cli
                                  /chronicle_credentials.json
  --v2                            Enable v2 commands.
  -h, --help                      Show this message and exit.
Sample output
Deleting Parser Extension…
ParserExtension deleted successfully.

deactivate_parser argument

Deactivates an active custom parser. If the deactivation is successful, you can start using the prebuilt parser for the given log type.

$ chronicle_cli parsers deactivate_parser -h
Usage: chronicle_cli parsers deactivate_parser [OPTIONS] PROJECT_ID CUSTOMER_ID
                                      LOG_TYPE PARSER_ID

  [New]Deactivate a parser

Options:
  --env [prod|test]               Optionally specify the environment for API
                                  calls.
  --region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|EUROPE-WEST12|ME-CENTRAL1|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
                                  Select region.
  --verbose                       Prints verbose output to the console.
  -c, --credential_file TEXT      Path of Service Account JSON. Default: /usr/
                                  local/home/<user>/.chronicle_cli
                                  /chronicle_credentials.json
  --v2                            Enable v2 commands.
  -h, --help                      Show this message and exit.
Sample output
Deactivating Parser…
Parser deactivated successfully.

activate_parser argument

Activates a custom parser. You can start using an active parser.

$ chronicle_cli parsers activate_parser -h
Usage: chronicle_cli parsers activate_parser [OPTIONS] PROJECT_ID CUSTOMER_ID LOG_TYPE
                                    PARSER_ID

  [New]Activate a parser

Options:
  --env [prod|test]               Optionally specify the environment for API
                                  calls.
  --region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|EUROPE-WEST12|ME-CENTRAL1|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
                                  Select region.
  --verbose                       Prints verbose output to the console.
  -c, --credential_file TEXT      Path of Service Account JSON. Default: /usr/
                                  local/home/<user>/.chronicle_cli
                                  /chronicle_credentials.json
  --v2                            Enable v2 commands.
  -h, --help                      Show this message and exit.
Sample output
Activating Parser…
Parser activated successfully.

get_parser argument

Fetches the details of the given parser ID and log type.

$ chronicle_cli parsers get_parser -h
Usage: chronicle_cli parsers get_parser [OPTIONS] PROJECT_ID CUSTOMER_ID LOG_TYPE
                               PARSER_ID

  [New]Get details of a parser

Options:
  -f, --file-format [TXT|JSON]    Format of the file to be exported.
  --export TEXT                   Export output to specified file path.
  --env [prod|test]               Optionally specify the environment for API
                                  calls.
  --region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|EUROPE-WEST12|ME-CENTRAL1|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
                                  Select region.
  --verbose                       Prints verbose output to the console.
  -c, --credential_file TEXT      Path of Service Account JSON. Default: /usr/
                                  local/home/<user>/.chronicle_cli
                                  /chronicle_credentials.json
  --v2                            Enable v2 commands.
  -h, --help                      Show this message and exit.
Sample output
Fetching Parser details...

Parser Details:
  Parser ID: 3840440184193679361
  Log type: GCP_CLOUDAUDIT
  State: INACTIVE
  Type: CUSTOM
  Author: -
  Validation Report ID: 3d2e1bdb-2793-48d1-a485-4f4748095cb8
  Create Time: 2023-04-14T09:15:13.718842Z

============================================================

get_extension argument

Fetches the configuration of the given parser ID and log type.

$ chronicle_cli parsers get_extension -h
Usage: chronicle_cli parsers get_extension [OPTIONS] PROJECT_ID CUSTOMER_ID LOG_TYPE
                                  PARSEREXTENSION_ID

  [New]Get details of an extension

Options:
  -f, --file-format [TXT|JSON]    Format of the file to be exported.
  --export TEXT                   Export output to specified file path.
  --env [prod|test]               Optionally specify the environment for API
                                  calls.
  --region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|EUROPE-WEST12|ME-CENTRAL1|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
                                  Select region.
  --verbose                       Prints verbose output to the console.
  -c, --credential_file TEXT      Path of Service Account JSON. Default: /usr/
                                  local/home/<user>/.chronicle_cli
                                  /chronicle_credentials.json
  --v2                            Enable v2 commands.
  -h, --help                      Show this message and exit.

Sample output
Fetching Parser Extension details...

ParserExtension Details:
  ParserExtension ID: 7b948bfb-d3f5-4922-9153-a20e75085990
  Log type: BRO_DNS
  State: VALIDATED
  Validation Report ID: 6ef30ad9-db89-4f30-80f3-0f79758ff3c2
  Create Time: 2023-07-06T03:58:26.594863Z
  State Last Changed Time: 2023-07-06T03:58:26.667151Z
  Last Live Time: 2023-07-06T03:58:28.019050Z

============================================================

get_validation_report argument

Fetches the validation report for a parser or an extension.

$ chronicle_cli parsers get_validation_report [OPTIONS] PROJECT_ID CUSTOMER_ID
                                          LOG_TYPE VALIDATION_REPORT_ID

  [New]Get validation report for a parser/extension

Options:
  --parser_id TEXT                ID of the parser.
  --parserextension_id TEXT       ID of the parser extension.
  --env [prod|test]               Optional: Specify the environment for API
                                  calls.
  --region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|EUROPE-WEST12|ME-CENTRAL1|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
                                  Select a region.
  --verbose                       Prints verbose output to the console.
  -c, --credential_file TEXT      Path of Service Account JSON. Default: /usr/
                                  local/google/home/<user>/.chronicle_cli
                                  /chronicle_credentials.json
  --v2                            Enable v2 commands.
  -h, --help                      Show this message and exit.
Sample output
Fetching Validation report for ParserExtension...

Validation Report:
  Verdict: PASS
  Stats:
    LogEntry Count: 10000
    Successfully Normalized Log Count: 10000
    Failed Log Count: 0
    Invalid Log Count: 0
    On Error Count: 153938
    Event Count: 10000
    Generic Event Count: 0
    Event Category:
      Valid_event: 10000
    Drop Tag:
      -
    Max Parse Duration: 0.274677769s
    Avg Parse Duration: 0.010s
    Normalization percent: 100
    Generic Event percent: 0
  Errors: -
  

Options

Help (-h / --help)

Use the -h or --help option to view usage with description for any command.

Sample Usage:
$ chronicle_cli parsers list_parsers -h
Usage: chronicle_cli parsers list_parsers [OPTIONS] PROJECT_ID CUSTOMER_ID LOG_TYPE

  [New]List all parsers for a given customer

Options:
  -s, --state [ALL|ACTIVE|INACTIVE]
                                  Filter on Parser State.
  -f, --file-format [TXT|JSON]    Format of the file to be exported.
  --export TEXT                   Export output to specified file path.
  --env [prod|test]               Optionally specify the environment for API
                                  calls.
  --region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|EUROPE-WEST12|ME-CENTRAL1|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
                                  Select region.
  --verbose                       Prints verbose output to the console.
  -c, --credential_file TEXT      Path of Service Account JSON. Default: /usr/
                                  local/home/<user>/.chronicle_cli
                                  /chronicle_credentials.json
  --v2                            Enable v2 commands.
  -h, --help                      Show this message and exit.

Environment (--env)

You can select the environment by passing the --env flag with command and the API calls will be executed accordingly. You can select an env value from the following: prod, test. If you do not specify this option, the default is set to prod.

Credentials File (-c or --credential_file)

This option allows to specify the path for the service account credentials to be used for authentication. If this option is not specified, then the Google SecOps CLI looks for credentials in the default path, i.e. ~/.chronicle_cli (inside a hidden directory named .chronicle_cli under the home directory).

Sample Usage:
$ chronicle_cli parsers list_parsers --credential_file=C:\chronicle_credentials.json

Region (--region)

You can select the region by passing the --region flag when you run a command. For more information on how to set the default region, see Set default region.

Verbose (--verbose)

This option allows you to print the details of the HTTP Request made and Response received.

Sample Usage:
$ chronicle_cli parsers list_parsers --verbose

Export (--export)

This option allows you to specify the file path where the output of the list or list_errors command will be exported. Both relative and absolute paths are supported.

Sample Usage:
$ chronicle_cli parsers list_parsers --export=parser_list

File Format (--file-format)

This option allows you to specify the file format of the contents exported with the list or list_errors command. Two formats are supported: JSON and TXT. If this option is not specified with the --export option, then TXT format is used as default.

Sample Usage:
$ chronicle_cli parsers list_parsers --export=parser_list --file-format=JSON

Parser management user workflows

The Google SecOps CLI can be used for managing CBN parsers using the following commands. However, it is recommended to use parser management v2 CLI commands.

Commands

parsers command

The parsers command takes these arguments:

  • archive
  • download
  • generate
  • history
  • list
  • list_errors
  • run
  • status
  • submit

Usage syntax:

$ chronicle_cli parsers ARGUMENT [OPTIONS]

Arguments

All CBN parser management workflows in the Google SecOps CLI are interactive. You are prompted for options when they are required for commands.

list argument

To list the details of all parsers, use the following command:

$ chronicle_cli parsers list -h
Usage: main parsers list [OPTIONS]

  List all parsers of a given customer

Options:
  --env [prod|test]               Optionally specify the environment for API
                                  calls.
  --region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|EUROPE-WEST12|ME-CENTRAL1|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
                                  Select region.
  --export TEXT                   Export output to specified file path.
  --file-format [TXT|CSV|JSON]    Format of the file to be exported.
  --verbose                       Prints verbose output to the console.
  -c, --credential_file TEXT      Path of Service Account JSON. Default: /usr/
                                  local/home/<user>/.chronicle_cli/chro
                                  nicle_credentials.json
  -h, --help                      Show this message and exit.
Sample output
Fetching list of parsers...

Parser Details:
  Config ID: 1cb402d9-eab2-4f6b-b402-20b1211675ed
  Log type: WINDOWS_SYSMON
  State: LIVE
  SHA256: 7xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx7
  Author: <user>@test.com
  Submit Time: 2022-08-26T09:57:10.644351Z
  State Last Changed Time: 2022-08-26T09:58:23.809636Z
  Last Live Time: 2022-08-26T09:58:23.809636Z
============================================================

Parser Details:
  Config ID: 7f2ae1f5-8f0c-43f9-bb02-299e7c8b9e82
  Log type: BOX
  State: LIVE
  SHA256: 8xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx7
  Author: <user>@test.com
  Submit Time: 2022-08-25T07:33:31.026399Z
  State Last Changed Time: 2022-08-25T07:33:32.263754Z
  Last Live Time: 2022-08-25T07:33:32.263754Z
============================================================

generate argument

To generate sample logs for a given log type, use the following command:

$ chronicle_cli parsers generate -h
Usage: main parsers generate [OPTIONS]

  Generate sample logs for a given log type

Options:
  --env [prod|test]               Optionally specify the environment for API
                                  calls.
  --region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|EUROPE-WEST12|ME-CENTRAL1|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
                                  Select region.
  -c, --credential_file TEXT      Path of Service Account JSON. Default: /usr/
                                  local/home/<user>/.chronicle_cli/chro
                                  nicle_credentials.json
  -h, --help                      Show this message and exit.

The command creates three files with 1, 10, and 1,000 sample logs at the root directory under <root>/chronicle_cli/cbn/<log_type>/.

Sample output
Enter Start Date (Format: yyyy-mm-ddThh:mm:ssZ): 2022-08-17T10:00:00Z
Enter End Date (Format: yyyy-mm-ddThh:mm:ssZ): 2022-08-23T10:00:00Z
Enter Log Type: WINDOWS_DHCP

Generating sample size: 1...
Generating sample size: 10...
Generating sample size: 1k...
Generated sample data (WINDOWS_DHCP); run this to go there:
cd /usr/local/home/<user>/cbn/windows_dhcp

history argument

To get the list of all parsers submission details for a given log type, use the following command:

$ chronicle_cli parsers history -h
Usage: main parsers history [OPTIONS]

  History retrieves all parsers submissions given a log type

Options:
  --env [prod|test]               Optionally specify the environment for API
                                  calls.
  --region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|EUROPE-WEST12|ME-CENTRAL1|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
                                  Select region.
  --verbose                       Prints verbose output to the console.
  -c, --credential_file TEXT      Path of Service Account JSON. Default: /usr/
                                  local/home/<user>/.chronicle_cli/chro
                                  nicle_credentials.json
  -h, --help                      Show this message and exit.
Sample output
Enter Log Type: WINDOWS_SYSMON
Fetching history for parser...

Parser History:
  Config ID: 8d9f5b1c-4689-4ca3-ae9b-863ce78dd123
  Log type: WINDOWS_SYSMON
  State: LIVE
  SHA256: 7xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx7
  Author: author@test.com
  Submit Time: 2022-08-26T12:37:55.187407Z
  State Last Changed Time: 2022-08-26T12:39:12.198587Z
  Last Live Time: 2022-08-26T12:39:12.198587Z

============================================================

Parser History:
  Config ID: 29bbf14b-2ffb-411a-bb37-911b13437123
  Log type: WINDOWS_SYSMON
  State: ARCHIVED
  SHA256: 8xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx7
  Author: author@test.com
  Submit Time: 2022-08-26T12:05:34.421743Z
  State Last Changed Time: 2022-08-26T12:39:12.198587Z
  Last Live Time: 2022-08-26T12:06:55.495269Z

============================================================

list_errors argument

To list errors of a log type between specific timestamps, use the following command:

$ chronicle_cli parsers list_errors -h
Usage: main parsers list_errors [OPTIONS]

  List errors of a log type between specific timestamps

Options:
  --env [prod|test]               Optionally specify the environment for API
                                  calls.
  --region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|EUROPE-WEST12|ME-CENTRAL1|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
                                  Select region.
  --export TEXT                   Export output to specified file path.
  --file-format [TXT|CSV|JSON]    Format of the file to be exported.
  --verbose                       Prints verbose output to the console.
  -c, --credential_file TEXT      Path of Service Account JSON. Default: /usr/
                                  local/home/<user>/.chronicle_cli/chro
                                  nicle_credentials.json
  -h, --help                      Show this message and exit.
Sample output
Enter Log Type: CISCO_ASA_FIREWALL
Enter Start Date (Format: yyyy-mm-ddThh:mm:ssZ): 2021-01-16T00:00:00Z
Enter End Date (Format: yyyy-mm-ddThh:mm:ssZ): 2022-08-21T12:00:00Z
Getting parser errors...
Error Details:
  Error ID: f9eb72cb-f320-dd5a-a098-00bcaa76a35d
  Config ID: N/A
  Log type: CISCO_ASA_FIREWALL
  Error Time: 2022-08-18T10:57:56.898883208Z
  Error Category: CBN_parsers_GENERATED_INVALID_EVENT
  Error Message: generic::invalid_argument: diff event timestamp ("seconds:1630106465") and create timestamp ("seconds:1660820265 nanos:202151000"): 8531h36m40.202151s, larger than allowed (4320h0m0s)
  Logs:
      <190>Aug 27 2020 23:21:05 TEST : %ASA-6-106012: Deny IP from 1.2.3.4 to 5.6.7.8, IP options: Test user
============================================================

Error Details:
  Error ID: f9eb72cb-f320-dd5a-a098-00bcaa76a35d
  Config ID: N/A
  Log type: CISCO_ASA_FIREWALL
  Error Time: 2022-08-18T10:57:56.898883208Z
  Error Category: CBN_parsers_GENERATED_INVALID_EVENT
  Error Message: generic::invalid_argument: diff event timestamp ("seconds:1630106465") and create timestamp ("seconds:1660820265 nanos:202151000"): 8531h36m40.202151s, larger than allowed (4320h0m0s)
  Logs:
      <190>Aug 27 2020 23:21:05 TEST : %ASA-6-106012: Deny IP from 1.2.3.4 to 5.6.7.8, IP options: Demo user

run argument

To validate the parser against given logs, use the following command:

$ chronicle_cli parsers run -h
Usage: main parsers run [OPTIONS]

  Run the parser against given logs

Options:
  --env [prod|test]               Optionally specify the environment for API
                                  calls.
  --region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|EUROPE-WEST12|ME-CENTRAL1|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
                                  Select region.
  --verbose                       Prints verbose output to the console.
  -c, --credential_file TEXT      Path of Service Account JSON. Default: /usr/
                                  local/home/<user>/.chronicle_cli/chro
                                  nicle_credentials.json
  -h, --help                      Show this message and exit.
Sample output
Enter path for conf file: /usr/local/home/Desktop/windows_sysmon.conf
Enter path for log file: /usr/local/home/Desktop/windows_sysmon.log
Running Validation…
Runtime: 2.4914s

submit argument

To submit a new parser, use the following command:

$ chronicle_cli parsers submit -h
Usage: main parsers submit [OPTIONS]

  Submit new parser

Options:
  --env [prod|test]               Optionally specify the environment for API
                                  calls.
  --region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|EUROPE-WEST12|ME-CENTRAL1|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
                                  Select region.
  --verbose                       Prints verbose output to the console.
  -c, --credential_file TEXT      Path of Service Account JSON. Default: /usr/
                                  local/home/<user>/.chronicle_cli/chro
                                  nicle_credentials.json
  -h, --help                      Show this message and exit.
Sample output
Enter Log type: CISCO_ASA_FIREWALL
Enter Config file path: /usr/local/Desktop/windows_sysmon.conf
Enter author: test
Submitting parser...

Submitted Parser Details:
  Config ID: 9ba20930-9733-4fcd-badf-18fedb9f8123
  Log type: CISCO_ASA_FIREWALL
  State: NEW
  SHA256: 7xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx7
  Author: test
  Submit Time: 2022-08-30T06:49:54.005119Z
  State Last Changed Time: 2022-08-30T06:49:54.005119Z

Parser submitted successfully. To get status of the parser, run this command using following Config ID - 9ba20930-9733-4fcd-badf-18fedb9f8123:
chronicle_cli parsers status

status argument

To get the status of a submitted parser, use the following command:

$ chronicle_cli parsers status -h
Usage: main parsers status [OPTIONS]

  Get status of a submitted parser

Options:
  --env [prod|test]               Optionally specify the environment for API
                                  calls.
  --region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|EUROPE-WEST12|ME-CENTRAL1|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
                                  Select region.
  --verbose                       Prints verbose output to the console.
  -c, --credential_file TEXT      Path of Service Account JSON. Default: /usr/
                                  local/home/<user>/.chronicle_cli/chro
                                  nicle_credentials.json
  -h, --help                      Show this message and exit.
Sample output
Enter Config ID: 1cb402d9-eab2-4f6b-b402-20b1211675ed

Getting parser...

Parser Details:
  Config ID: 1cb402d9-eab2-4f6b-b402-20b1211675ed
  Log type: WINDOWS_SYSMON
  State: ARCHIVED
  SHA256: 79ac67c15ffb047a152be2fb2a3391cbe18b2d183e9e6a402eb2fe53a6666b17
  Author: test
  Submit Time: 2022-08-26T09:57:10.644351Z
  State Last Changed Time: 2022-08-26T09:58:23.809636Z
  Last Live Time: 2022-08-26T09:58:23.809636Z

archive argument

To archive an existing parser, use the following command:

$ chronicle_cli parsers archive -h
Usage: main parsers archive [OPTIONS]

  Archives a parser given the config ID.

Options:
  --env [prod|test]               Optionally specify the environment for API
                                  calls.
  --region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|EUROPE-WEST12|ME-CENTRAL1|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
                                  Select region.
  --verbose                       Prints verbose output to the console.
  -c, --credential_file TEXT      Path of Service Account JSON. Default: /usr/
                                  local/home/<user>/.chronicle_cli/chro
                                  nicle_credentials.json
  -h, --help                      Show this message and exit.
Sample output
Enter Config ID: 1cb402d9-eab2-4f6b-b402-20b121167123
Archiving parser...

Parser archived Successfully.

Parser Details:
  Config ID: 1cb402d9-eab2-4f6b-b402-20b121167123
  Log type: WINDOWS_SYSMON
  State: ARCHIVED
  SHA256: 7xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx7
  Author: test
  Submit Time: 2022-08-26T09:57:10.644351Z
  State Last Changed Time: 2022-08-26T09:58:23.809636Z
  Last Live Time: 2022-08-26T09:58:23.809636Z

download argument

To download the configuration (.conf) file for a given log type or config ID, use the following command:

$ chronicle_cli parsers download -h
Usage: main parsers download [OPTIONS]

  Download parser code by given Config ID or Log type.

Options:
  --env [prod|test]               Optionally specify the environment for API
                                  calls.
  --region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|EUROPE-WEST12|ME-CENTRAL1|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
                                  Select region.
  --verbose                       Prints verbose output to the console.
  -c, --credential_file TEXT      Path of Service Account JSON. Default: /usr/
                                  local/home/<user>/.chronicle_cli/chro
                                  nicle_credentials.json
  -h, --help                      Show this message and exit.
Sample output
  • Using Config ID.
Note: If you want to download parser by log type then skip the config ID.
Enter config ID: 9d1474ab-eff2-4855-ba57-4f0c458e3ac2
Downloading parser...
Writing parser to: CISCO_ASA_FIREWALL_20220825131911.conf
  • Using Log Type.
Note: If you want to download parser by log type then skip the config ID.
Enter config ID:
Enter Log Type: CISCO_ASA_FIREWALL
Downloading parser...
Writing parser to: CISCO_ASA_FIREWALL_20220825132011.conf

Options

Help (-h / --help)

Use the -h or --help option to view usage with description for any command.

Sample Usage:
$ chronicle_cli parsers list -h
Usage: main parsers list [OPTIONS]

  List all parsers of a given customer

Options:
  -f, --file-format [TXT|JSON]    Format of the file to be exported.
  --export TEXT                   Export output to specified file path.
  --env [prod|test]               Optionally specify the environment for API
                                  calls.
  --region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|EUROPE-WEST12|ME-CENTRAL1|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
                                  Select region.
  --verbose                       Prints verbose output to the console.
  -c, --credential_file TEXT      Path of Service Account JSON. Default: /usr/
                                  local/home/<user>/.chronicle_cli/chro
                                  nicle_credentials.json
  -h, --help                      Show this message and exit.

Environment (--env)

You can select the environment by passing the --env flag with command and the API calls will be executed accordingly. You can select an env value from the following: prod, test. If you don't specify this option, the default is set to prod.

Credentials File (-c or --credential_file)

This option allows to specify the path for the service account credentials to be used for the authentication of the user. If this option is not specified, then the Google SecOps CLI looks for credentials in the default path, i.e. ~/.chronicle_cli (inside a hidden directory named .chronicle_cli under the home directory).

Sample Usage:
$ chronicle_cli parsers list --credential_file=C:\chronicle_credentials.json

Region (--region)

You can select the region by passing the --region flag when you run a command. For more information on how to set the default region, see Set default region.

Verbose (--verbose)

This option allows you to print the details of the HTTP Request made and Response received.

Sample Usage:
$ chronicle_cli parsers list --verbose

Export (--export)

This option allows you to specify the file path where the output of the list or list_errors command will be exported. Both relative and absolute paths are supported.

Sample Usage:
$ chronicle_cli parsers list --export=parsers_list

File Format (--file-format)

This option allows you to specify the file format of the contents exported with the list or list_errors command. There are three formats supported: JSON and TXT. If this option is not specified with the --export option, then TXT format would be used as default.

Sample Usage:
$ chronicle_cli parsers list --export=parsers_list --file-format=JSON

Troubleshooting

CBN Parser Management Error Codes

This section shows the output displayed on the console against different types of response code received from API response.

Refer to below table to view output for each command:

Response Code Command Console Output
Any response code other than 200 {command}[archive, download, generate, history, list_errors, run, status, list, submit] Error while {command} parser.
Response Code: {status code}
Error: {error message}

Forwarder management user workflows

The Google SecOps CLI can be used to manage forwarders and their associated collectors using the following commands:

Commands

forwarders command

The forwarders command takes these arguments:

  • create
  • update
  • get
  • list
  • delete
  • generate_files
  • collectors

Usage syntax:

$ chronicle_cli forwarders ARGUMENT [OPTIONS]

collectors command

The collectors command takes these arguments:

  • create
  • update
  • get
  • list
  • delete

Usage syntax:

$ chronicle_cli forwarders collectors ARGUMENT [OPTIONS]

Arguments

All Forwarder Management user workflows in the Google SecOps CLI are interactive. You are prompted for options as and when required.

create argument

To create a new forwarder and configure collectors for it, use the following command:

$ chronicle_cli forwarders create --help
Usage: main forwarders create [OPTIONS]

  Create a Forwarder

Options:
  --url TEXT                      Base URL to be used for API calls.
  --region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|EUROPE-WEST12|ME-CENTRAL1|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
                                  Select region.
  --verbose                       Prints verbose output to the console.
  -c, --credential_file TEXT      Path of Service Account JSON. Default: /usr/
                                  local/google/home/<user>/.chronicle_cl
                                  i/chronicle_credentials.json
  -h, --help                      Show this message and exit.
Sample output
$ chronicle_cli forwarders create
================================================================================
Press Enter if you want to use the default value mentioned besides field description in [] brackets.
================================================================================

(*) Forwarder Display Name : test_display_name

========================================
======== Forwarder Configuration =======
========================================

Upload Compression (Determines if uploaded data will be compressed) [Y/n]: y

Do you want to proceed with Forwarder Metadata? [y/N]: y

========================================
========== Forwarder Metadata ==========
========================================

Asset Namespace: test_namespace

========================================
=========== Forwarder Labels ===========
========================================

Labels (The ingestion metadata labels in 'key:value' format to apply to all logs ingested through this forwarder, as well as the resulting normalized data.)
Enter/Paste your content. On a new line, press Ctrl-D (Linux) / [Ctrl-Z + Enter (Windows)] to save it:
key1:value1
key2:value2

Do you want to proceed with Forwarder Regex Filters? [y/N]: y

========================================
======= Forwarder Regex Filters =======
========================================

Filter Description (Describes what is being filtered and why): desc1

Filter Regexp (The regular expression used to match against each incoming line): .*

Filter Behavior (Filter behavior to apply when a match is found)
Choose:
1. allow
2. block
: 1

You have selected allow

Do you want to add more Forwarder Regex Filters [y/N]: y

Filter Description (Describes what is being filtered and why): desc2

Filter Regexp (The regular expression used to match against each incoming line): .*

Filter Behavior (Filter behavior to apply when a match is found)
Choose:
1. allow
2. block
: 2

You have selected block

Do you want to add more Forwarder Regex Filters [y/N]: n

Do you want to proceed with Server Settings? [y/N]: y

========================================
=========== Server Settings ===========
========================================

Server State (Server State for Collector)
Choose:
1. active
2. suspended
: 1

You have selected active

Graceful Timeout (Number of seconds after which the forwarder returns a bad readiness/health check and still accepts new connections) [15]:

Drain timeout (Number of seconds after which the forwarder waits for active connections to successfully close on their own before being closed by the server) [10]:

Do you want to proceed with HTTP-specific server settings? [y/N]: y

========================================
==== HTTP-specific server settings ====
========================================

Host (IP address, or hostname that can be resolved to IP addresses, that the server should listen on) [0.0.0.0]: 10.0.14.132

Port (Port number that the HTTP server listens on for health checks from the load balancer) [8080]: 8000

Read Timeout (Maximum amount of time allowed to read the entire request, both the header and the body) [3]:

Read Header Timeout (Maximum amount of time allowed to read request headers) [3]:

Write Timeout (Maximum amount of time allowed to send a response) [3]:

Idle Timeout (Maximum amount of time (in seconds) to wait for the next request when idle connections are enabled) [3]:

Do you want to proceed with Route Settings? [y/N]: y

========================================
============ Route Settings ============
========================================

Available Status Code (Status code returned when a liveness check is received and the forwarder is available) [204]: 200

Ready Status Code (Status code returned when it is ready to accept traffic) [204]: 200

Unready Status Code (Status code returned when it is not ready to accept traffic) [503]: 500

Preview changes:

  - Press Up/b or Down/z keys to paginate.
  - To switch case-sensitivity, press '-i' and press enter. By default, search
    is case-sensitive.
  - To search for specific field, press '/' key, enter text and press enter.
  - Press 'q' to quit and confirm preview changes.
  - Press `h` for all the available options to navigate the list.
=============================================================================

Config:
  Metadata:
    Asset namespace: test_namespace
    Labels:
    - key: key1
      value: value1
    - key: key2
      value: value2
  Regex filters:
  - behavior: ALLOW
    description: desc1
    regexp: .*
  - behavior: BLOCK
    description: desc2
    regexp: .*
  Server settings:
    Drain timeout: 10
    Graceful timeout: 15
    Http settings:
      Host: 10.0.14.132
      Idle timeout: 3
      Port: 8000
      Read header timeout: 3
      Read timeout: 3

Do you want to create forwarder with this configuration [y/N]: y

Creating forwarder...
Forwarder created successfully with Forwarder ID: a7e59660-959b-44e7-aa7e-baec820d01f4

Would you like to configure collectors for this forwarder? [y/N]: y

(*) Collector Display Name: collector_1

========================================
======== Collector Configuration ======
========================================

(*) Collector Log Type (Type of logs collected): WINDOWS_DNS

Do you want to proceed with Collector Metadata? [y/N]: y

========================================
========== Collector Metadata ==========
========================================

Asset Namespace: test_namespace

========================================
=========== Forwarder Labels ===========
========================================

Labels (The ingestion metadata labels in 'key:value' format to apply to all logs ingested through this forwarder, as well as the resulting normalized data.)
Enter/Paste your content. On a new line, press Ctrl-D (Linux) / [Ctrl-Z + Enter (Windows)] to save it:
key1:value1
key2:value2

Do you want to proceed with Collector Regex Filters? [y/N]: y

========================================
======= Collector Regex Filters =======
========================================

Filter Description (Describes what is being filtered and why): desc1

Filter Regexp (The regular expression used to match against each incoming line): .*

Filter Behavior (Filter behavior to apply when a match is found)
Choose:
1. allow
2. block
: 1

You have selected allow

Do you want to add more Collector Regex Filters [y/N]: n

Do you want to proceed with Collector Disk Buffer? [y/N]: y

========================================
======== Collector Disk Buffer ========
========================================

Disk Buffer State (Disk buffering state for collector)
Choose:
1. active
2. suspended
: 1

You have selected active

Directory Path (Directory path for files written): path/to/file.txt

Max File Buffer Bytes (Maximum buffered file size): 45

Maximum Seconds per Batch (Maximum number of seconds between forwarder batch uploads) [10]:

Maximum Bytes per Batch (Maximum number of bytes queued before forwarder batch upload) [1048576]:

========================================
===== Configure Ingestion Settings =====
========================================

Choose:
1. File Settings
2. Kafka Settings
3. Pcap Settings
4. Splunk Settings
5. Syslog Settings
: 1

File Path (Path of file to monitor): path/to/file.txt

Preview changes:

  - Press Up/b or Down/z keys to paginate.
  - To switch case-sensitivity, press '-i' and press enter. By default, search
    is case-sensitive.
  - To search for specific field, press '/' key, enter text and press enter.
  - Press 'q' to quit and confirm preview changes.
  - Press `h` for all the available options to navigate the list.
=============================================================================

Config:
  Disk buffer:
    Directory path: path/to/file.txt
    Max file buffer bytes: 45
    State: ACTIVE
  File settings:
    File path: path/to/file.txt
  Log type: WINDOWS_DNS
  Max bytes per batch: 1048576
  Max seconds per batch: 10
  Metadata:
    Asset namespace: test_namespace
    Labels:
    - key: key1
      value: value1
    - key: key2
      value: value2
  Regex filters:
    Behavior: ALLOW
    Description: desc1
    Regexp: .*
Display name: collector_1

Do you want to create collector with this configuration [y/N]: y

Creating collector...
Collector created successfully with Collector ID: 1f72f9ab-3ae3-4c5f-955e-86c982587937

Would you like to add more collectors? [y/N]: n

If the forwarder create fails, and you enter the same forwarder ID, then you are prompted to retry the failed forwarder or restart the process. If the forwarder ID entered by you does not match with the failed forwarder ID, then you will not be prompted to retry and the normal process of creating the forwarder continues.

Sample output
...
Creating forwarder...

Error occurred while creating forwarder.
Response Code: 500.
Error: ZERO_APP::1: create forwarder due to validation errors in request: generic::invalid_argument: filter's description is not specified

$ chronicle_cli forwarders create

Looks like there was a failed create/update attempt for test.
Would you like to retry?


(*) Forwarder Display Name [test]:

Do you want to create forwarder with this configuration [y/N]: y

Creating forwarder...
Forwarder created successfully with Forwarder ID: ab7af569-d957-44a3-99a8-aa70ffdc6458

Would you like to configure collectors for this forwarder? [y/N]: n

get argument

To get details of an existing forwarder and respective collectors, use the following command:

$ chronicle_cli forwarders get --help
Usage: main forwarders get [OPTIONS]

  Get forwarder details using Forwarder ID

Options:
  --url TEXT                      Base URL to be used for API calls.
  --region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|EUROPE-WEST12|ME-CENTRAL1|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
                                  Select region.
  --verbose                       Prints verbose output to the console.
  -c, --credential_file TEXT      Path of Service Account JSON. Default: /usr/
                                  local/google/home/<user>/.chronicle_cl
                                  i/chronicle_credentials.json
  -h, --help                      Show this message and exit.
Sample output

You must enter the forwarder ID to get the forwarder details.

$ chronicle_cli forwarders get
Enter Forwarder ID: a7e59660-959b-44e7-aa7e-baec820d01f4

Fetching forwarder and its all associated collectors...

Forwarder Details:

ID: a7e59660-959b-44e7-aa7e-baec820d01f4
Display name: test_display_name
State: ACTIVE
Config:
  Upload compression: true
  Metadata:
    Asset namespace: test_namespace
    Labels:
    - key: key1
      value: value1
    - key: key2
      value: value2
  Regex filters:
  - description: desc1
    regexp: .*
    behavior: ALLOW
  - description: desc2
    regexp: .*
    behavior: BLOCK
  Server settings:
    Graceful timeout: 15
    Drain timeout: 10
    Http settings:
      Port: 8000
      Host: 10.0.14.132
      Read timeout: 3
      Read header timeout: 3
      Write timeout: 3
      Idle timeout: 3
      Route settings:
        Available status code: 200
        Ready status code: 200
        Unready status code: 500
    State: ACTIVE

Collectors:
  Collector [1f72f9ab-3ae3-4c5f-955e-86c982587937]:
    Display name: collector_1
    State: ACTIVE
    Config:
      Log type: WINDOWS_DNS
      Metadata:
        Asset namespace: test_namespace
        Labels:
        - key: key1
          value: value1
        - key: key2
          value: value2
      Regex filters:
      - description: desc1
        regexp: .*
        behavior: ALLOW
      Disk buffer:
        State: ACTIVE
        Directory path: path/to/file.txt
        Max file buffer bytes: '45'
      Max seconds per batch: 10
      Max bytes per batch: '1048576'
      File settings:
        File path: path/to/file.txt

================================================================================

list argument

To display list of all forwarders and respective collectors, use following command:

$ chronicle_cli forwarders list --help
Usage: main forwarders list [OPTIONS]

  List all forwarders

Options:
  --url TEXT                      Base URL to be used for API calls.
  --region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|EUROPE-WEST12|ME-CENTRAL1|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
                                  Select region.
  --verbose                       Prints verbose output to the console.
  --export TEXT                   Export output to specified file path
  --file-format [TXT|CSV|JSON]    Format of the file to be exported
  -c, --credential_file TEXT      Path of Service Account JSON. Default: /usr/
                                  local/google/home/<user>/.chronicle_cl
                                  i/chronicle_credentials.json
  -h, --help                      Show this message and exit.

Command is used to fetch the details of all the forwarders and collectors.

Sample output
$ chronicle_cli forwarders list
Fetching list of forwarders...

Forwarder Details:

ID: a7e59660-959b-44e7-aa7e-baec820d01f4
Display name: test_display_name
State: ACTIVE
Config:
  Upload compression: true
  Metadata:
    Asset namespace: test_namespace
    Labels:
    - key: key1
      value: value1
    - key: key2
      value: value2
  Regex filters:
  - description: desc1
    regexp: .*
    behavior: ALLOW
  - description: desc2
    regexp: .*
    behavior: BLOCK
  Server settings:
    Graceful timeout: 15
    Drain timeout: 10
    Http settings:
      Port: 8000
      Host: 10.0.14.132
      Read timeout: 3
      Read header timeout: 3
      Write timeout: 3
      Idle timeout: 3
      Route settings:
        Available status code: 200
        Ready status code: 200
        Unready status code: 500
    State: ACTIVE

Collectors:
  Collector [1f72f9ab-3ae3-4c5f-955e-86c982587937]:
    Display name: collector_1
    State: ACTIVE
    Config:
      Log type: WINDOWS_DNS
      Metadata:
        Asset namespace: test_namespace
        Labels:
        - key: key1
          value: value1
        - key: key2
          value: value2
      Regex filters:
      - description: desc1
        regexp: .*
        behavior: ALLOW
      Disk buffer:
        State: ACTIVE
        Directory path: path/to/file.txt
        Max file buffer bytes: '45'
      Max seconds per batch: 10
      Max bytes per batch: '1048576'
      File settings:
        File path: path/to/file.txt

================================================================================

Forwarder Details:

ID: ddcca884-cdc6-4ac2-ad30-05a28e6cf35a
Display name: test
State: ACTIVE
Config:
  Upload compression: true
  Metadata:
    Asset namespace: test
    Labels:
    - key: k1
      value: v2
  Regex filters:
  - description: hh
    regexp: hh
    behavior: ALLOW
  - description: gg
    regexp: gg
    behavior: BLOCK
  Server settings:
    Graceful timeout: 15
    Drain timeout: 10
    Http settings:
      Port: 8080
      Host: 0.0.0.0
      Read timeout: 3
      Read header timeout: 3
      Write timeout: 3
      Idle timeout: 3
      Route settings:
        Available status code: 204
        Ready status code: 204
        Unready status code: 503
    State: ACTIVE

Collectors:
  Message: No collectors found for this forwarder.

================================================================================

If you want to export the data, then you can specify the absolute/relative path of the file to be exported along with the format of the file (CSV/TXT/JSON). The default file format is CSV.

Sample output
$ chronicle_cli forwarders list --export=$HOME/listforwarder --file-format=JSON
Fetching list of forwarders...

Forwarder Details:

ID: a7e59660-959b-44e7-aa7e-baec820d01f4
Display name: test_display_name
State: ACTIVE
Config:
  Upload compression: true
  Metadata:
    Asset namespace: test_namespace
    Labels:
    - key: key1
      value: value1
    - key: key2
      value: value2
  Regex filters:
  - description: desc1
    regexp: .*
    behavior: ALLOW
  - description: desc2
    regexp: .*
    behavior: BLOCK
  Server settings:
    Graceful timeout: 15
    Drain timeout: 10
    Http settings:
      Port: 8000
      Host: 10.0.14.132
      Read timeout: 3
      Read header timeout: 3
      Write timeout: 3
      Idle timeout: 3
      Route settings:
        Available status code: 200
        Ready status code: 200
        Unready status code: 500
    State: ACTIVE

Collectors:
  Collector [1f72f9ab-3ae3-4c5f-955e-86c982587937]:
    Display name: collector_1
    State: ACTIVE
    Config:
      Log type: WINDOWS_DNS
      Metadata:
        Asset namespace: test_namespace
        Labels:
        - key: key1
          value: value1
        - key: key2
          value: value2
      Regex filters:
      - description: desc1
        regexp: .*
        behavior: ALLOW
      Disk buffer:
        State: ACTIVE
        Directory path: path/to/file.txt
        Max file buffer bytes: '45'
      Max seconds per batch: 10
      Max bytes per batch: '1048576'
      File settings:
        File path: path/to/file.txt

================================================================================

Forwarder Details:

ID: ddcca884-cdc6-4ac2-ad30-05a28e6cf35a
Display name: test
State: ACTIVE
Config:
  Upload compression: true
  Metadata:
    Asset namespace: test
    Labels:
    - key: k1
      value: v2
  Regex filters:
  - description: hh
    regexp: hh
    behavior: ALLOW
  - description: gg
    regexp: gg
    behavior: BLOCK
  Server settings:
    Graceful timeout: 15
    Drain timeout: 10
    Http settings:
      Port: 8080
      Host: 0.0.0.0
      Read timeout: 3
      Read header timeout: 3
      Write timeout: 3
      Idle timeout: 3
      Route settings:
        Available status code: 204
        Ready status code: 204
        Unready status code: 503
    State: ACTIVE

Collectors:
  Message: No collectors found for this forwarder.

================================================================================
Forwarders list details exported successfully to: /usr/local/google/home/<user>/listforwarder.json

update argument

To update an existing forwarder, use the following command:

$ chronicle_cli forwarders update --help
Usage: main forwarders update [OPTIONS]

  Update a forwarder using forwarder ID.

Options:
  --url TEXT                      Base URL to be used for API calls.
  --region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|EUROPE-WEST12|ME-CENTRAL1|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
                                  Select region.
  --verbose                       Prints verbose output to the console.
  -c, --credential_file TEXT      Path of Service Account JSON. Default: /usr/
                                  local/google/home/<user>/.chronicle_cl
                                  i/chronicle_credentials.json
  -h, --help                      Show this message and exit.

After running the command, enter the Forwarder ID and all the field values again. You can reuse the old values by pressing the Enter.

Sample output
$ chronicle_cli forwarders update
Enter Forwarder ID: a7e59660-959b-44e7-aa7e-baec820d01f4
Press Enter if you don't want to update.

(*) Forwarder Display Name [test_display_name]:

========================================
======== Forwarder Configuration =======
========================================

Upload Compression (Determines if uploaded data will be compressed) [Y/n]: y

Do you want to proceed with Forwarder Metadata? [y/N]: y

========================================
========== Forwarder Metadata ==========
========================================

Asset Namespace [test_namespace]:

========================================
=========== Forwarder Labels ===========
========================================

Labels (The ingestion metadata labels in 'key:value' format to apply to all logs ingested through this forwarder, as well as the resulting normalized data.)
Enter/Paste your content. On a new line, press Ctrl-D (Linux) / [Ctrl-Z + Enter (Windows)] to save it:
[[{'key': 'key1', 'value': 'value1'}, {'key': 'key2', 'value': 'value2'}]]

Do you want to proceed with Forwarder Regex Filters? [y/N]: n

Do you want to proceed with Server Settings? [y/N]: n

Do you want to update forwarder with this configuration? [y/N]: y

Updating forwarder...

Forwarder updated successfully with Forwarder ID: a7e59660-959b-44e7-aa7e-baec820d01f4

If the forwarder update fails and you enter the same forwarder, then a prompt is displayed to retry the failed forwarder or to restart the process. If you enter a forwarder ID which does not match the failed forwarder ID, then the option to retry is not displayed and the normal process of updating the forwarder continues. The retry mechanism allows you to change option values provided in the previous failed attempt in an interactive way. Press Enter to reuse the same value for an option in the forwarder update flow.

delete argument

Use this argument to delete a forwarder using a forwarder ID. On execution, it asks for the ID of the feed to be deleted. To delete an existing forwarder, use the following command:

chronicle_cli forwarders delete --help
Usage: main forwarders delete [OPTIONS]

  Delete a forwarder using Forwarder ID

Options:
  --url TEXT                      Base URL to be used for API calls.
  --region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|EUROPE-WEST12|ME-CENTRAL1|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
                                  Select region.
  --verbose                       Prints verbose output to the console.
  -c, --credential_file TEXT      Path of Service Account JSON. Default: /usr/
                                  local/google/home/<user>/.chronicle_cl
                                  i/chronicle_credentials.json
  -h, --help                      Show this message and exit.
Sample output
$ chronicle_cli forwarders delete
Enter Forwarder ID: 0593ba21-a1c7-4279-b429-bc8df959bd59

Deleting forwarder and all its associated collectors...

Forwarder (ID: 0593ba21-a1c7-4279-b429-bc8df959bd59) deleted successfully with all its associated collectors.

generate_files argument

Use this argument to generate file having information about forwarder using forwarder ID.

To generate the forwarder file, use the following command:

$ chronicle_cli forwarders generate_files -h
Usage: main forwarders generate_files [OPTIONS]

  Generate forwarder configuration files using Forwarder ID

Options:
  --url TEXT                      Base URL to be used for API calls.
  --region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|EUROPE-WEST12|ME-CENTRAL1|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
                                  Select region.
  --verbose                       Prints verbose output to the console.
  -c, --credential_file TEXT      Path of Service Account JSON. Default: /usr/
                                  local/google/home/<user>/.chronicle_cl
                                  i/chronicle_credentials.json
  -f, --file-path TEXT            Download generated forwarder files to the
                                  specified path.
  -h, --help                      Show this message and exit.
Sample output
$ chronicle_cli forwarders generate_files --file-path=$HOME/GenerateForwarderFile
Enter Forwarder ID: 0768220e-8af6-4ef7-a1dd-73e33963b444
Generating forwarder files ...
Forwarder files generated successfully.
Configuration file: /usr/local/google/home/<user>/GenerateForwarderFile_forwarder.conf
Auth file: /usr/local/google/home/<user>/GenerateForwarderFile_forwarder_auth.conf

Collector sub commands

create argument

Use the following command to configure a new collector for a given forwarder.

$ chronicle_cli forwarders collectors create --help
Usage: main forwarders collectors create [OPTIONS]

  Create a collector.

Options:
  --url TEXT                      Base URL to be used for API calls.
  --region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|EUROPE-WEST12|ME-CENTRAL1|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
                                  Select region.
  --verbose                       Prints verbose output to the console.
  -c, --credential_file TEXT      Path of Service Account JSON. Default: /usr/
                                  local/google/home/<user>/.chronicle_cl
                                  i/chronicle_credentials.json
  -h, --help                      Show this message and exit.
Sample output
$ chronicle_cli forwarders collectors create
================================================================================
Press Enter if you want to use the default value mentioned besides field description in [] brackets.
================================================================================

Enter Forwarder ID: a7e59660-959b-44e7-aa7e-baec820d01f4

(*) Collector Display Name: collector_4

========================================
======== Collector Configuration ======
========================================

(*) Collector Log Type (Type of logs collected): WINDOWS_DNS

Do you want to proceed with Collector Metadata? [y/N]: y

========================================
========== Collector Metadata ==========
========================================

Asset Namespace: test_namespace

========================================
=========== Forwarder Labels ===========
========================================

Labels (The ingestion metadata labels in 'key:value' format to apply to all logs ingested through this forwarder, as well as the resulting normalized data.)
Enter/Paste your content. On a new line, press Ctrl-D (Linux) / [Ctrl-Z + Enter (Windows)] to save it:
key1:value1
key2:value2

Do you want to proceed with Collector Regex Filters? [y/N]: y

========================================
======= Collector Regex Filters =======
========================================

Filter Description (Describes what is being filtered and why): desc1

Filter Regexp (The regular expression used to match against each incoming line): .*

Filter Behavior (Filter behavior to apply when a match is found)
Choose:
1. allow
2. block
: 1

You have selected allow

Do you want to add more Collector Regex Filters? [y/N]: n

Do you want to proceed with Collector Disk Buffer? [y/N]: n

Maximum Seconds per Batch (Maximum number of seconds between forwarder batch uploads) [10]:

Maximum Bytes per Batch (Maximum number of bytes queued before forwarder batch upload) [1048576]:

========================================
===== Configure Ingestion Settings =====
========================================

Choose:
1. File Settings
2. Kafka Settings
3. Pcap Settings
4. Splunk Settings
5. Syslog Settings


File Path (Path of file to monitor): path/to/file.txt

Preview changes:

  - Press Up/b or Down/z keys to paginate.
  - To switch case-sensitivity, press '-i' and press enter. By default, search
    is case-sensitive.
  - To search for specific field, press '/' key, enter text and press enter.
  - Press 'q' to quit and confirm preview changes.
  - Press `h` for all the available options to navigate the list.
=============================================================================

Config:
  File settings:
    File path: path/to/file.txt
  Log type: WINDOWS_DNS
  Max bytes per batch: 1048576
  Max seconds per batch: 10
  Metadata:
    Asset namespace: test_namespace
    Labels:
    - key: key1
      value: value1
    - key: key2
      value: value2
  Regex filters:
    Behavior: ALLOW
    Description: desc1
    Regexp: .*
Display name: collector_4

Do you want to create collector with this configuration? [y/N]: y

Creating collector...
Collector created successfully with Collector ID: 3a74b289-ccb4-4cee-9713-611a3362f48f

If the collector create fails, and you enter the same collector ID, then a prompt is displayed to retry the failed collector or restart the process. If you enter the collector ID which does not match with the failed collector ID, then the option to retry is not displayed and the normal process of updating the collector continues.

Sample output
$ chronicle_cli forwarders collectors create
Enter Forwarder ID: a7e59660-959b-44e7-aa7e-baec820d01f4

Looks like there was a failed create/update attempt for test_display.
Would you like to retry?


(*) Collector Display Name [test_display]:

========================================
======== Collector Configuration ======
========================================

(*) Collector Log Type (Type of logs collected) [WINDOWS_DNS]:

Do you want to proceed with Collector Metadata? [y/N]: y

========================================
========== Collector Metadata ==========
========================================

Asset Namespace [test]:

========================================
=========== Forwarder Labels ===========
========================================

Labels (The ingestion metadata labels in 'key:value' format to apply to all logs ingested through this forwarder, as well as the resulting normalized data.)
Enter/Paste your content. On a new line, press Ctrl-D (Linux) / [Ctrl-Z + Enter (Windows)] to save it:
[[{'key': 'k1', 'value': 'v1'}]]

Do you want to proceed with Collector Regex Filters? [y/N]: y

========================================
======= Collector Regex Filters =======
========================================

Filter Description (Describes what is being filtered and why)[old_desc]: desc2

Filter Regexp (The regular expression used to match against each incoming line) [.*]:

Filter Behavior (Filter behavior to apply when a match is found)
Choose:
1. allow
2. block


You have selected allow

Do you want to add more Collector Regex Filters? [y/N]: n

Do you want to proceed with Collector Disk Buffer? [y/N]: n

Maximum Seconds per Batch (Maximum number of seconds between forwarder batch uploads) [10]:

Maximum Bytes per Batch (Maximum number of bytes queued before forwarder batch upload) [1048576]:

========================================
===== Configure Ingestion Settings =====
========================================

Choose:
1. File Settings
2. Kafka Settings
3. Pcap Settings
4. Splunk Settings
5. Syslog Settings
: 1

File Path (Path of file to monitor) [path/to/file.txt]: path/to/file.txt

Preview changes:

  - Press Up/b or Down/z keys to paginate.
  - To switch case-sensitivity, press '-i' and press enter. By default, search
    is case-sensitive.
  - To search for specific field, press '/' key, enter text and press enter.
  - Press 'q' to quit and confirm preview changes.
  - Press `h` for all the available options to navigate the list.
=============================================================================

Config:
  File settings:
    File path: path/to/file.txt
  Log type: WINDOWS_DNS
  Max bytes per batch: 1048576
  Max seconds per batch: 10
  Metadata:
    Asset namespace: test
    Labels:
    - key: k1
      value: v1
  Regex filters:
    Behavior: ALLOW
    Description: disc2
    Regexp: .*
Display name: test_display

Do you want to create collector with this configuration? [y/N]: y

Creating collector...
Collector created successfully with Collector ID: b50a6b41-5476-41ee-ba7c-ce529ecffa62

get argument

To get details of an existing collectors, use the following command:

$ chronicle_cli forwarders collectors get --help
Usage: main forwarders collectors get [OPTIONS]

  Get a collector using collector ID.

Options:
  --url TEXT                      Base URL to be used for API calls.
  --region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|EUROPE-WEST12|ME-CENTRAL1|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
                                  Select region.
  --verbose                       Prints verbose output to the console.
  -c, --credential_file TEXT      Path of Service Account JSON. Default: /usr/
                                  local/google/home/<user>/.chronicle_cl
                                  i/chronicle_credentials.json
  -h, --help                      Show this message and exit.
Sample output

You must enter the collector ID interactively to get the collector details.

$ chronicle_cli forwarders collectors get
Enter Forwarder ID: a7e59660-959b-44e7-aa7e-baec820d01f4
Enter Collector ID: 3a74b289-ccb4-4cee-9713-611a3362f48f

Fetching collector details...

Collector Details:

ID: 3a74b289-ccb4-4cee-9713-611a3362f48f
Display name: collector_4
State: ACTIVE
Config:
  Log type: WINDOWS_DNS
  Metadata:
    Asset namespace: test_namespace
    Labels:
    - key: key1
      value: value1
    - key: key2
      value: value2
  Regex filters:
  - description: desc1
    regexp: .*
    behavior: ALLOW
  - description: desc2
    regexp: .*
    behavior: BLOCK
  Max seconds per batch: 10
  Max bytes per batch: '1048576'
  File settings:
    File path: path/to/file.txt

list argument

To display list of all collectors, use following command:

chronicle_cli forwarders collectors list --help
Usage: main forwarders collectors list [OPTIONS]

  List all collectors.

Options:
  --url TEXT                      Base URL to be used for API calls.
  --region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|EUROPE-WEST12|ME-CENTRAL1|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
                                  Select region.
  --verbose                       Prints verbose output to the console.
  --export TEXT                   Export output to specified file path.
  --file-format [TXT|CSV|JSON]    Format of the file to be exported.
  -c, --credential_file TEXT      Path of Service Account JSON. Default: /usr/
                                  local/google/home/<user>/.chronicle_cl
                                  i/chronicle_credentials.json
  -h, --help                      Show this message and exit.
Sample output
$ chronicle_cli forwarders collectors list
Enter Forwarder ID: a7e59660-959b-44e7-aa7e-baec820d01f4

Collector Details:

ID: 153e4077-cd49-4ce5-87aa-254d239b9dda
Display name: collector_2
State: ACTIVE
Config:
  Log type: WINDOWS_DNS
  Metadata:
    Asset namespace: test
    Labels:
    - key: key1
      value: value1
    - key: key2
      value: value2
  Regex filters:
  - description: desc1
    regexp: .*
    behavior: ALLOW
  Disk buffer:
    State: ACTIVE
    Directory path: path/to/dir
    Max file buffer bytes: '209'
  Max seconds per batch: 10
  Max bytes per batch: '1048576'
  File settings:
    File path: path/to/file.txt

================================================================================

Collector Details:

ID: b50a6b41-5476-41ee-ba7c-ce529ecffa62
Display name: test_display
State: ACTIVE
Config:
  Log type: WINDOWS_DNS
  Metadata:
    Asset namespace: test
    Labels:
    - key: k1
      value: v1
  Regex filters:
  - description: disc2
    regexp: .*
    behavior: ALLOW
  - description: test
    regexp: test
    behavior: BLOCK
  Disk buffer:
    State: ACTIVE
    Directory path: test
    Max file buffer bytes: '55'
  Max seconds per batch: 5
  Max bytes per batch: '556676'
  Syslog settings:
    Protocol: TCP
    Address: 1.2.3.4
    Port: 3456
    Buffer size: '65536'
    Connection timeout: 60
    Tls settings:
      Certificate: test
      Certificate key: test
      Minimum tls version: '56'
      Insecure skip verify: true

================================================================================

update argument

To update an existing collector, use the following command:

$ chronicle_cli forwarders collectors update --help
Usage: main forwarders collectors update [OPTIONS]

  Update a collector using collector ID.

Options:
  --url TEXT                      Base URL to be used for API calls.
  --region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|EUROPE-WEST12|ME-CENTRAL1|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
                                  Select region.
  --verbose                       Prints verbose output to the console.
  -c, --credential_file TEXT      Path of Service Account JSON. Default: /usr/
                                  local/google/home/<user>/.chronicle_cl
                                  i/chronicle_credentials.json
  -h, --help                      Show this message and exit.
Sample output
$ chronicle_cli forwarders collectors update
Enter Forwarder ID: a7e59660-959b-44e7-aa7e-baec820d01f4
Enter Collector ID: 3a74b289-ccb4-4cee-9713-611a3362f48f

(*) Collector Display Name [collector_4]:

========================================
======== Collector Configuration ======
========================================

(*) Collector Log Type (Type of logs collected) [WINDOWS_DNS]:

Do you want to proceed with Collector Metadata? [y/N]: y

========================================
========== Collector Metadata ==========
========================================

Asset Namespace [test_namespace]:

========================================
=========== Forwarder Labels ===========
========================================

Labels (The ingestion metadata labels in 'key:value' format to apply to all logs ingested through this forwarder, as well as the resulting normalized data.)
Enter/Paste your content. On a new line, press Ctrl-D (Linux) / [Ctrl-Z + Enter (Windows)] to save it:
[{'key1':'value1'},{'key2':'value2'}]

Do you want to proceed with Collector Regex Filters? [y/N]: y

========================================
======= Collector Regex Filters =======
========================================

Filter Description (Describes what is being filtered and why)[old_desc]: desc1

Filter Regexp (The regular expression used to match against each incoming line)[.*]: .*

Filter Behavior (Filter behavior to apply when a match is found)
Choose:
1. allow
2. block


You have selected allow

Do you want to add more Collector Regex Filters? [y/N]: y

Filter Description (Describes what is being filtered and why): desc2

Filter Regexp (The regular expression used to match against each incoming line): .*

Filter Behavior (Filter behavior to apply when a match is found)
Choose:
1. allow
2. block
: 2

You have selected block

Do you want to add more Collector Regex Filters? [y/N]: n

Do you want to proceed with Collector Disk Buffer? [y/N]: n

Maximum Seconds per Batch (Maximum number of seconds between forwarder batch uploads) [10]:

Maximum Bytes per Batch (Maximum number of bytes queued before forwarder batch upload) [1048576]:

========================================
===== Configure Ingestion Settings =====
========================================

Choose:
1. File Settings
2. Kafka Settings
3. Pcap Settings
4. Splunk Settings
5. Syslog Settings


File Path (Path of file to monitor) [path/to/file.txt]: path/to/file.txt

Do you want to update collector with this configuration? [y/N]: y

Updating collector...

Collector updated successfully with Collector ID: 3a74b289-ccb4-4cee-9713-611a3362f48f

If the collector update fails and you enter the same collector ID, then a prompt is displayed to retry the failed collector or to restart the process. If you enter a collector ID which does not match the failed collector ID, then the option to retry is not displayed and the normal process of updating the collector continues. The retry mechanism allows you to change option values provided in the previous failed attempt in an interactive way. Press Enter to reuse the same value for an option in the collector update flow.

...
Updating collector...

Do you want to update collector with this configuration? [y/N]: y

Error occurred while updating collector.
Response Code: 400.
Error: generic::invalid_argument: update collector (id: 3a74b289-ccb4-4cee-9713-611a3362f48f) for forwarder (id: a7e59660-959b-44e7-aa7e-baec820d01f4) for customer (id: ed19f037-2354-43df-bfbf-350362b45844): validation errors in request: generic::invalid_argument: filter's description is not specified: invalid argument

$ chronicle_cli forwarders collectors update
Enter Forwarder ID: a7e59660-959b-44e7-aa7e-baec820d01f4
Enter Collector ID: 3a74b289-ccb4-4cee-9713-611a3362f48f

Looks like there was a failed create/update attempt for collector_4.
Would you like to retry?


(*) Collector Display Name [collector_4]:

========================================
======== Collector Configuration ======
========================================

(*) Collector Log Type (Type of logs collected) [WINDOWS_DNS]:

Do you want to proceed with Collector Metadata? [y/N]: n

Do you want to proceed with Collector Regex Filters? [y/N]: y

========================================
======= Collector Regex Filters =======
========================================

Filter Description (Describes what is being filtered and why)[old_desc]: desc1

Filter Regexp (The regular expression used to match against each incoming line) [.*]:

Filter Behavior (Filter behavior to apply when a match is found)
Choose:
1. allow
2. block


You have selected allow

Do you want to add more Collector Regex Filters? [y/N]: n

Do you want to proceed with Collector Disk Buffer? [y/N]: n

Maximum Seconds per Batch (Maximum number of seconds between forwarder batch uploads) [10]:

Maximum Bytes per Batch (Maximum number of bytes queued before forwarder batch upload) [1048576]:

========================================
===== Configure Ingestion Settings =====
========================================

Choose:
1. File Settings
2. Kafka Settings
3. Pcap Settings
4. Splunk Settings
5. Syslog Settings
[1]:

File Path (Path of file to monitor) [path/to/file.txt]:

Do you want to update collector with this configuration? [y/N]: y

Updating collector...

Collector updated successfully with Collector ID: 3a74b289-ccb4-4cee-9713-611a3362f48f

delete argument

Use this argument to delete a collector using a collector ID. On execution, it asks for the ID of the collector to be deleted.

To delete an existing collector, use the following command:

$ chronicle_cli forwarders collectors delete --help
Usage: main forwarders collectors delete [OPTIONS]

  Delete a collector using collector ID.

Options:
  --url TEXT                      Base URL to be used for API calls.
  --region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|EUROPE-WEST12|ME-CENTRAL1|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
                                  Select region.
  --verbose                       Prints verbose output to the console.
  -c, --credential_file TEXT      Path of Service Account JSON. Default: /usr/
                                  local/google/home/<user>/.chronicle_cl
                                  i/chronicle_credentials.json
  -h, --help                      Show this message and exit.
Sample output
$ chronicle_cli forwarders collectors delete
Enter Forwarder ID: a7e59660-959b-44e7-aa7e-baec820d01f4
Enter Collector ID: 3a74b289-ccb4-4cee-9713-611a3362f48f

Collector (ID: 3a74b289-ccb4-4cee-9713-611a3362f48f) deleted successfully.

Options

Help (-h / --help)

Use -h or --help option to view usage/description for any command/option.

Sample Usage
$ chronicle_cli forwarders list -h
Usage: main forwarders list [OPTIONS]

  List all forwarders

Options:
  --url TEXT                      Base URL to be used for API calls.
  --region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|EUROPE-WEST12|ME-CENTRAL1|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
                                  Select region.
  --verbose                       Prints verbose output to the console.
  --export TEXT                   Export output to specified file path.
  --file-format [TXT|CSV|JSON]    Format of the file to be exported.
  -c, --credential_file TEXT      Path of Service Account JSON. Default: /usr/
                                  local/google/home/<user>/.chronicle_cl
                                  i/chronicle_credentials.json
  -h, --help                      Show this message and exit.

Credentials Path (-c or --credential-path)

This option allows to specify the path for the service account credentials to be used for the authentication of the user. If this option is not specified, then the Google SecOps CLI looks for credentials in the default path, i.e. ~/.chronicle_cli (inside a hidden directory named .chronicle_cli under the home directory).

Sample Usage
$ chronicle_cli forwarders list --credential-path=C:\chronicle_credentials.json

Verbose (--verbose)

This option allows you to print the details of the HTTP Request made and Response received.

Sample Usage
$ chronicle_cli forwarders list --verbose

Export (--export)

This option allows you to specify the file path where the output of the list command will be exported. Both relative and absolute paths are supported.

Sample Usage
$ chronicle_cli forwarders list --export=$HOME/listForwarderssResponse

File Format (--file-format)

This option allows you to specify the file format of the contents exported with the list command. There are three formats supported: CSV, JSON and TXT. If this option is not specified with the --export option, then CSV format would be used as default.

Sample Usage
$ chronicle_cli forwarders list --export=$HOME/listForwardersResponse --file-format=JSON
Sample output
JSON format
{
  "forwarders": [
    {
      "name": "55a77e24-9d16-4638-8940-0ef8071ed849",
      "displayName": "new",
      "config": {
        "uploadCompression": true,
        "metadata": {
          "assetNamespace": "test",
          "labels": [
            {
              "key": "k",
              "value": "v"
            },
            {
              "key": "k1",
              "value": "v1"
            }
          ]
        },
        "regexFilters": [
          {
            "description": "desc1",
            "regexp": ".*",
            "behavior": "ALLOW"
          }
        ],
        "serverSettings": {
          "gracefulTimeout": 15,
          "drainTimeout": 10,
          "httpSettings": {
            "port": 8080,
            "host": "0.0.0.0",
            "readTimeout": 3,
            "readHeaderTimeout": 3,
            "writeTimeout": 3,
            "idleTimeout": 3,
            "routeSettings": {
              "availableStatusCode": 204,
              "readyStatusCode": 204,
              "unreadyStatusCode": 503
            }
          },
          "state": "ACTIVE"
        }
      },
      "state": "ACTIVE",
      "collectors": {
        "Collector [3e8243c3-7ff2-4ede-89fe-16410ffe03bd]": {
          "name": "3e8243c3-7ff2-4ede-89fe-16410ffe03bd",
          "displayName": "cre_test_2",
          "state": "ACTIVE",
          "config": {
            "logType": "WINDOWS_DNS",
            "metadata": {
              "assetNamespace": "test",
              "labels": [
                {
                  "key": "k",
                  "value": "v"
                }
              ]
            },
            "regexFilters": [
              {
                "description": "desc1",
                "regexp": ".*",
                "behavior": "ALLOW"
              }
            ],
            "diskBuffer": {
              "state": "ACTIVE",
              "directoryPath": "23",
              "maxFileBufferBytes": "33"
            },
            "maxSecondsPerBatch": 10,
            "maxBytesPerBatch": "1048576",
            "fileSettings": {
              "filePath": "path/file.txt"
            }
          }
        }
      }
    }
  ]
}
CSV format

1. {file_name}_forwarders.csv

2. {file_name}_collectors.csv

Sample content of file:

{file_name}_forwarders.csv:

Name,Display name,Forwarder state,[CONFIG] Upload compression,[CONFIG][METADATA] Asset namespace,[CONFIG][METADATA] Labels,[CONFIG] Regex filters,[CONFIG][SERVER_SETTINGS] Server state,[CONFIG][SERVER_SETTINGS] Graceful timeout,[CONFIG][SERVER_SETTINGS] Drain timeout,[CONFIG][SERVER_SETTINGS][HTTP_SETTINGS] Port,[CONFIG][SERVER_SETTINGS][HTTP_SETTINGS] Host,[CONFIG][SERVER_SETTINGS][HTTP_SETTINGS] Read timeout,[CONFIG][SERVER_SETTINGS][HTTP_SETTINGS] Read header timeout,[CONFIG][SERVER_SETTINGS][HTTP_SETTINGS] Write timeout,[CONFIG][SERVER_SETTINGS][HTTP_SETTINGS] Idle timeout,[CONFIG][SERVER_SETTINGS][HTTP_SETTINGS][ROUTE_SETTINGS] Available status code,[CONFIG][SERVER_SETTINGS][HTTP_SETTINGS][ROUTE_SETTINGS] Ready status code,[CONFIG][SERVER_SETTINGS][HTTP_SETTINGS][ROUTE_SETTINGS] Unready status code
0593ba21-a1c7-4279-b429-bc8df959bd59,test,ACTIVE,True,test,"k1: v1

k2: v2
",,,,,,,,,,,0,0,0
094c9e41-e7c8-407a-8b9a-eb34d608a609,test,ACTIVE,True,te,"k1: v1

k2: v2
",,,,,,,,,,,0,0,0
1189f869-5f3c-4ec7-ba48-9c80e33aadf0,test,ACTIVE,True,test,"key1: value1

key2: value2
",,ACTIVE,15,10,8080,0.0.0.0,3,4,5,8,204,204,500

{file_name}_collectors.csv

Forwarder ID,Name,Display Name,Collector state,[CONFIG] Log type,[CONFIG] Max seconds per batch,[CONFIG] Max bytes per batch,[CONFIG][METADATA] Asset namespace,[CONFIG][METADATA] Labels,[CONFIG] Regex filters,[CONFIG][DISK_BUFFER] State,[CONFIG][DISK_BUFFER] Directory path,[CONFIG][DISK_BUFFER] Max file buffer bytes,[CONFIG][FILE_SETTINGS] File path,[CONFIG][KAFKA_SETTINGS][AUTHENTICATION] username,[CONFIG][KAFKA_SETTINGS][AUTHENTICATION] password,[CONFIG][KAFKA_SETTINGS] Topic,[CONFIG][KAFKA_SETTINGS] Group id,[CONFIG][KAFKA_SETTINGS] Timeout,[CONFIG][KAFKA_SETTINGS] Brokers,[CONFIG][KAFKA_SETTINGS][TLS_SETTINGS] Certificate,[CONFIG][KAFKA_SETTINGS][TLS_SETTINGS] Certificate key,[CONFIG][KAFKA_SETTINGS][TLS_SETTINGS] Minimum tls version,[CONFIG][KAFKA_SETTINGS][TLS_SETTINGS] Insecure skip verify,[CONFIG][PCAP_SETTINGS] Network interface,[CONFIG][PCAP_SETTINGS] Bpf,[CONFIG][SPLUNK_SETTINGS][AUTHENTICATION] username,[CONFIG][SPLUNK_SETTINGS][AUTHENTICATION] Password,[CONFIG][SPLUNK_SETTINGS] Host,[CONFIG][SPLUNK_SETTINGS] Port,[CONFIG][SPLUNK_SETTINGS] Minimum window size,[CONFIG][SPLUNK_SETTINGS] Maximum windows size,[CONFIG][SPLUNK_SETTINGS] Query string,[CONFIG][SPLUNK_SETTINGS] Query mode,[CONFIG][SPLUNK_SETTINGS] Cert ignored,[CONFIG][SYSLOG_SETTINGS] Protocol,[CONFIG][SYSLOG_SETTINGS] Address,[CONFIG][SYSLOG_SETTINGS] Port,[CONFIG][SYSLOG_SETTINGS] Buffer size,[CONFIG][SYSLOG_SETTINGS] Connection timeout,[CONFIG][SYSLOG_SETTINGS][TLS_SETTINGS] Certificate,[CONFIG][SYSLOG_SETTINGS][TLS_SETTINGS] Certificate key,[CONFIG][SYSLOG_SETTINGS][TLS_SETTINGS] Minimum tls version,[CONFIG][SYSLOG_SETTINGS][TLS_SETTINGS] Insecure skip verify
1189f869-5f3c-4ec7-ba48-9c80e33aadf0,03d28371-1bcb-4b28-9364-18412de1f827,collector_2,ACTIVE,WINDOWS_DNS,10,1048576,collector_update,"key1: value1

key2: value2
",,ACTIVE,path/file.txt,23,path/to/file.txt,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
1189f869-5f3c-4ec7-ba48-9c80e33aadf0,8ba8278c-1eef-4a72-a45a-491463768c70,col_3,ACTIVE,WINDOWS_DNS,10,1048576,test,"k1: v1
",,ACTIVE,path/to/file,233,path,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
55a77e24-9d16-4638-8940-0ef8071ed849,3e8243c3-7ff2-4ede-89fe-16410ffe03bd,cre_test_2,ACTIVE,WINDOWS_DNS,10,1048576,test,"k: v
",,ACTIVE,23,33,path/file.txt,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,

Region (--region)

You can select the region by passing the --region flag when you run a command. For more information on how to set the default region, see Set default region.

Troubleshooting

Error Codes

This section shows the output displayed on the console against different types of response codes received from API response.

Refer to the following tables to view the output for each command:

Get command

Response Code Console Output
404 {Forwarder|Collector} does not exist.
400 Invalid {Forwarder|Collector} ID. Please enter valid {Forwarder|Collector} ID.
Any other response code Error while fetching {Forwarder|Collector}. Response Code: {status code} Error: {error message}

List command

Response Code Console Output
Any response code other than 200 Error: Response Code: {status code} Error: {error message}

Create command

Response Code Console Output
Any response code other than 200 Error occurred while creating {forwarder|collector} Response Code: {status code} Error: {error message}

Update command

Response Code Console Output
Any response code other than 200 Error occurred while updating {forwarder|collector}. Response Code: {status code} Error: {error message}

Delete command

Response Code Console Output
404 {Forwarder|Collector} does not exist.
400 Invalid {Forwarder|Collector} ID. Please enter valid {Forwarder|Collector} ID.
Any other response code Error occurred while deleting {Forwarder|Collector}. Response Code: {status code} Error: {error message}

Other errors or exceptions

Exception Console Output
KeyError Failed to find key {key name} in the response.
Exception Failed with exception: {exception details}
Credential file missing Failed with exception: [Errno 2] No such file or directory: '/usr/local/google/home//.chronicle_cli/chronicle_credentials.json'

You need to place credentials in the expected directory. Refer to Installation.

For any other queries or issues with the Google Security Operations CLI, please reach out to Google Security Operations Support.

BigQuery data access workflows

Google Security Operations supports self-service access to Google Security Operations data (both SIEM and SOAR) in BigQuery. You can use the Google Security Operations CLI to grant Identity and Access Management (IAM) roles that give the following permissions for a user email:

  • roles/bigquery.dataViewer
  • roles/bigquery.jobUser
  • roles/storage.objectViewer

The email must be a Google Accounts and ID Administration (GAIA) user email address of a Google SecOps customer.

For more information about these roles, see Exporting table data.

Commands

bigquery command

The bigquery command takes the provide_access argument.

Usage syntax:

$ chronicle_cli bigquery ARGUMENT [OPTIONS]

Arguments

provide_access argument

Prompts you to enter a user email address. The email must be a Google Accounts and ID Administration (GAIA) user email address of a Google SecOps Security customer. The user will receive the required IAM roles to do the following:

  • Read data and metadata from BigQuery tables (roles/bigquery.dataViewer)
  • Run queries on BigQuery table data (roles/bigquery.jobUser)
  • Read data in Google Cloud Storage buckets(roles/storage.objectViewer)
Sample usage
$ chronicle_cli bigquery provide_access
$ Enter email: xyz@gmail.com
Success response
Providing BigQuery access...
Access provided to email: xyz@gmail.com
Error response
Providing BigQuery access...
Error while providing access:
Response code: 400

Options

Help (-h / --help)

Use -h or --help option to view usage/description for any command/option.

Troubleshooting

This section shows the output displayed on the console against different types of response codes received from API response.

provide_access argument response codes

Response Code Console Output
400 Email does not exist.
Any other response code Error while fetching feed. Response Code: {status code} Error: {error message}