Google Security Operations CLI User Guide
As an alternative to using the Google Security Operations graphical user interface, advanced users can use the Google Security Operations command-line interface (CLI), chronicle_cli, to initiate Google Security Operations workflows.
You can use the Google Security Operations CLI for these workflows:
Google Security Operations CLI commands use the following syntax (COMMAND and ARGUMENT are required, but OPTIONS is optional):
$ chronicle_cli COMMAND ARGUMENT [OPTIONS]
For example, to create a new feed using the feed management workflows, use the following command:
$ chronicle_cli feeds create
Before you begin
Before installing Google Security Operations CLI, do the following:
- Install Python 3 in your environment. For more information, see Installing Python.
- Create a virtual environment and activate it. For more information, see Installing packages using pip and virtual environments.
- Create a hidden directory named
.chronicle_cliin your home directory and place your service account credential inside it with the namechronicle_credentials.json. For more information, see Create and manage service account keys. If you plan to use the parser management v2 commands, you must do the following:
- Bind your Google Security Operations instance to a project that you own. See the following documents:
- Create a service account in the project bound to Google Security Operations.
- Grant the service account the Chronicle API Admin (
roles/chronicle.admin) role.
Installation
This section provides information on how to install the Google Security Operations CLI in your environment.
Create and activate the virtual environment
venv.Clone the repository using following command:
git clone https://github.com/chronicle/cli.gitOpen a terminal and install all the required dependent packages in your virtual environment by executing the following command:
$ cd cli $ (env) pip install -r requirements.txtInstall the Google Security Operations binary by executing the following command:
$ (env) python3 -m pip install --editable .Verify that the installation is successful by executing the following command:
$ chronicle_cli --help
Sample output
Usage: chronicle_cli [OPTIONS] COMMAND [ARGS]...
Google Security Operations CLI is a CLI tool for managing Google Security Operations user workflows for e.g.
Feed Management workflows.
Options:
-h, --help Show this message and exit.
Commands:
feeds Feed Management Workflows
Options
You can override the default configuration by providing additional flags when you run commands.
Set a default region (--region)
You can select a region by passing the --region flag with the command and
the API calls will be made to the appropriate Google Security Operations region backend.
You can set the following regions:
ASIA-NORTHEAST1ASIA-SOUTH1ASIA-SOUTHEAST1AUSTRALIA-SOUTHEAST1EUROPEEUROPE-WEST2EUROPE-WEST3EUROPE-WEST6ME-CENTRAL2ME-WEST1NORTHAMERICA-NORTHEAST2US
If you don't specify the region, the default region is set to US.
Feed management workflows
You can use the Google Security Operations CLI to create and manage data feeds to your Google SecOps instance.
Commands
feeds command
The feeds command takes these arguments:
createupdategetlistdeleteenabledisable
Usage syntax:
$ chronicle_cli feeds ARGUMENT [OPTIONS]
Arguments
create argument
Creates a new feed.
Sample usage
$ chronicle_cli feeds create --help
Usage: chronicle_cli feeds create [OPTIONS]
Create a feed
Options:
--url TEXT Base URL to be used for API calls.
--region
Select region.
--verbose Prints verbose output to the console.
-c, --credential-path TEXT Path of Service Account JSON. Default: /usr/
local/google/home/<user>/.chronicle_cli/chronicle_credentials.json
-h, --help Show this message and exit.
Sample output
- For Windows platform
====================================
========== Set Properties ==========
====================================
List of Source types:
1. Amazon S3
2. Amazon SQS
3. Google Cloud Storage
4. HTTP(S) URI
5. Microsoft Azure Blob Storage
6. SFTP
7. Third party API
[Source type] Enter your choice: 7
You have selected Third party API
List of Log types:
(i) How to select log type?
- Press ENTER key (scrolls one line at a time) or SPACEBAR key (display next screen).
- Note down the choice number for the log type that you want to select.
- Press 'q' to quit and enter that choice number.
=============================================================================
1. Anomali
2. Azure AD
3. Azure AD Directory Audit
4. Azure AD Organizational Context
5. Cloud Passage
6. Duo Auth
7. Duo User Context
8. Fox-IT
9. Imperva
10. Microsoft Graph API Alerts:
[Log type] Enter your choice: 7
You have selected Duo User Context
Enter feed display name: my_duo_user_context_feed
======================================
=========== Input Parameters =========
======================================
(*) - Required fields.
Password/secret inputs are hidden.
(*) Username (Username to authenticate as)
=> USERNAME
(*) Secret (Secret to authenticate with)
=>
(*) API hostname (The fully qualified domain name for your instance of the API, having the form "api-xxxxxxxx.duosecurity.com")
=> api-xxxxxxxx.duosecurity.com
Feed created successfully with Feed ID: 9cfce415-97df-413b-8e38-e7c747f9ed38
- For other platforms such as Linux/Ubuntu/CentOS/MacOS
====================================
========== Set Properties ==========
====================================
List of Source types:
1. Amazon S3
2. Amazon SQS
3. Google Cloud Storage
4. HTTP(S) URI
5. Microsoft Azure Blob Storage
6. SFTP
7. Third party API
[Source type] Enter your choice: 7
You have selected Third party API
List of Log types:
(i) How to select log type?
- Press Up/b or Down/z keys to paginate.
- To switch case-sensitivity, press '-i' and press enter. By default, search
is case-sensitive.
- To search for specific log type, press '/' key, enter text and press enter.
- Note down the choice number for the log type that you want to select.
- Press 'q' to quit and enter that choice number.
- Press `h` for all the available options to navigate the list.
=============================================================================
1. Anomali
2. Azure AD
3. Azure AD Directory Audit
4. Azure AD Organizational Context
5. Cloud Passage
6. Duo Auth
7. Duo User Context
8. Fox-IT
9. Imperva:
[Log type] Enter your choice: 7
You have selected Duo User Context
======================================
======================================
(*) - Required fields.
Password/secret inputs are hidden.
(*) Username (Username to authenticate as)
=> USERNAME
(*) Secret (Secret to authenticate with)
=>
(*) API hostname (The fully qualified domain name for your instance of the API, having the form "api-xxxxxxxx.duosecurity.com")
=> api-xxxxxxxx.duosecurity.com
Feed created successfully with Feed ID: 9cfce415-97df-413b-8e38-e7c747f9ed38
If feed creation fails, the next time you are asked to retry the failed feed creation. You can opt to retry or to continue creating a new feed. The retry mechanism allows you to interactively change values provided in the previous failed attempt. Press Enter to reuse the same value for an option in the feed creation flow.
Sample output
====================================
========== Set Properties ==========
====================================
List of Source types:
1. Amazon S3
2. Amazon SQS
3. Google Cloud Storage
4. HTTP(S) URI
5. Microsoft Azure Blob Storage
6. SFTP
7. Third party API
[Source type] Enter your choice: 7
You have selected Third party API
[Log type] Enter your choice: 6
You have selected Duo Auth
Enter feed display name: my_duo_auth_feed
======================================
=========== Input Parameters =========
======================================
(*) - Required fields.
Password/secret inputs are hidden.
(*) Username (Username to authenticate as)
=> test
(*) Secret (Secret to authenticate with)
=>
(*) API hostname (The fully qualified domain name for your instance of the API, having the form "api-xxxxxxxx.duosecurity.com")
=> test.com
Error occurred while creating feed.
Response Code: 400.
Error: generic::invalid_argument: failed to create feed for the customer (ID: ed19f037-2354-43df-bfbf-350362b45844): failed to create feed for the customer (ID: ed19f037-2354-43df-bfbf-350362b45844): failed to create feed because of the following errors in the request: generic::invalid_argument: for Duo feeds, 'hostname' must be specified as "api-xxxxxxxx.duosecurity.com", e.g. "api-eval.duosecurity.com"
$ chronicle_cli feeds create
Looks like there was a failed feed create/update attempt with source type: Third party API and log type: Duo Auth.
Would you like to retry?
======================================
=========== Input Parameters =========
======================================
(*) - Required fields.
Password/secret inputs are hidden.
(*) Username (Username to authenticate as) [test]
=>
(*) Secret (Secret to authenticate with)
=>
(*) API hostname (The fully qualified domain name for your instance of the API, having the form "api-xxxxxxxx.duosecurity.com") [test.com]
=> api-xxxxxxxx.duosecurity.com
Feed created successfully with Feed ID: 29a2f967-6f6e-4521-bebf-6fb6c7383df6
get argument
Gets details of an existing feed.
Sample usage
$ chronicle_cli feeds get --help
Usage: main feeds get [OPTIONS]
Get feed details using Feed ID
Options:
--url TEXT Base URL to be used for API calls.
--region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
Select region.
--verbose Prints verbose output to the console.
-c, --credential-path TEXT Path of Service Account JSON. Default: /usr/
local/google/home/<user>/.chronicle_cli/chronicle_credentials.json
-h, --help Show this message and exit.
You must enter the feed ID interactively to get the feed details.
Sample output
Enter Feed ID: 72d9b843-b387-4b17-ab2d-a8497313c89c
Feed Details:
ID: 72d9b843-b387-4b17-ab2d-a8497313c89c
Display Name: my_duo_auth_feed
Source type: Third party API
Log type: Salesforce
State: ACTIVE
Feed Settings:
API Hostname: myinstance.salesforce.com
list argument
Lists all feeds. This command is used to fetch the details of all the feeds.
Sample usage
$ chronicle_cli feeds list --help
Usage: chronicle_cli feeds list [OPTIONS]
List all feeds
Options:
--url TEXT Base URL to be used for API calls.
--region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
Select region.
--verbose Prints verbose output to the console.
--export TEXT Export output to specified file path.
--file-format [TXT|CSV|JSON] Format of the file to be exported.
-c, --credential-path TEXT Path of Service Account JSON. Default: /usr/
local/google/home/<user>/.chronicle_cli/chronicle_credentials.json
-h, --help Show this message and exit.
Sample output
Feed Details:
ID: 29259301-156b-4b60-ae91-855d15c39f6a
Source type: Third party API
Log type: Anomali
State: INACTIVE
============================================================
Feed Details:
ID: 292b7629-0250-476c-9fb2-4c8a738ce42c
Display Name: my_duo_auth_feed
Source type: Third party API
Log type: Duo Auth
State: ACTIVE
Feed Settings:
API hostname: api-test.duosecurity.com
============================================================
Feed Details:
ID: 0d063a7f-34a1-4dd0-9dcf-9c7a0bb03e65
Source type: Third party API
Log type: Workspace Activities
State: ACTIVE
Feed Settings:
Customer ID: C12abc
Applications: ['drive', 'login']
============================================================
If you want to export the data, then you can specify the absolute/relative path of the file to be exported along with the format of the file (CSV/TXT/JSON). The default file format is CSV.
Sample output
Feed Details:
ID: 29259301-156b-4b60-ae91-855d15c39f6a
Source type: Third party API
Log type: Anomali
State: INACTIVE
============================================================
Feed Details:
ID: 292b7629-0250-476c-9fb2-4c8a738ce42c
Display Name: my_duo_auth_feed
Source type: Third party API
Log type: Duo Auth
State: ACTIVE
Feed Settings:
API hostname: api-test.duosecurity.com
============================================================
Feed Details:
ID: 0d063a7f-34a1-4dd0-9dcf-9c7a0bb03e65
Source type: Third party API
Log type: Workspace Activities
State: ACTIVE
Feed Settings:
Customer ID: C12abc
Applications: ['drive', 'login']
============================================================
Feed list details exported successfully to: /usr/local/google/home/<user>/out/chronicle-cli/output.txt
update argument
Updates an existing feed.
Sample usage
$ chronicle_cli feeds update
Usage: chronicle_cli feeds update [OPTIONS]
Update feed details using Feed ID
Options:
--url TEXT Base URL to be used for API calls.
--region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
Select region.
--verbose Prints verbose output to the console.
-c, --credential-path TEXT Path of Service Account JSON. Default: /usr/
local/google/home/<user>/.chronicle_cli/chronicle_credentials.json
-h, --help Show this message and exit.
After running the command, enter a feed ID and all the field values again. Press Enter to reuse the old value.
Sample output
Enter Feed ID: ea28d66b-d81b-4b4d-ae16-3b1cd98132ca
Press Enter if you don't want to update.
Enter feed display name[old_display_name]:
(*) Username (Username to authenticate as)
=> USERNAME
(*) Secret (Secret to authenticate with)
=>
(*) API hostname (The fully qualified domain name for your instance of the API, having the form "api-xxxxxxxx.duosecurity.com") [api-xxxxxxxx.duosecurity.com]
=>
Feed updated successfully with Feed ID: ea28d66b-d81b-4b4d-ae16-3b1cd98132ca
Enter Feed ID: 29a2f967-6f6e-4521-bebf-6fb6c7383df6
Press Enter if you don't want to update.
Enter feed display name[]: my_feed_display_name
(*) Username (Username to authenticate as)
=> test1
(*) Secret (Secret to authenticate with)
=>
(*) API hostname (The fully qualified domain name for your instance of the API, having the form "api-xxxxxxxx.duosecurity.com") [api-xxxxxxxx.duosecurity.com]
=> test.com
Error occurred while updating feed. Response code: 400.
Error: generic::invalid_argument: failed to update feed for the customer (ID: ed19f037-2354-43df-bfbf-350362b45844): failed to edit feed because of the following errors in the request: generic::invalid_argument: for Duo feeds, 'hostname' must be specified as "api-xxxxxxxx.duosecurity.com", e.g. "api-eval.duosecurity.com"
$ chronicle_cli feeds update
Enter Feed ID: 29a2f967-6f6e-4521-bebf-6fb6c7383df6
Looks like there was a failed feed create/update attempt with source type: Third party API and log type: Duo Auth.
Would you like to retry?
Press Enter if you don't want to update.
(*) Username (Username to authenticate as) [test1]
=>
(*) Secret (Secret to authenticate with)
=>
(*) API hostname (The fully qualified domain name for your instance of the API, having the form "api-xxxxxxxx.duosecurity.com") [test.com]
=> api-devtest.duosecurity.com
Feed updated successfully with Feed ID: 29a2f967-6f6e-4521-bebf-6fb6c7383df6
If the feed update fails and you enter the same feed ID, then a prompt is displayed to retry the failed feed or to restart the process. If you enter a feed ID which does not match the failed feed ID, then the option to retry is not displayed and the normal process of updating the feed continues. The retry mechanism allows you to change option values provided in the previous failed attempt in an interactive way. Press Enter to reuse the same value for an option in the feed update flow.
Sample output
Enter Feed ID: 51574667-dee6-408b-a5fc-0e07d3e9a429
Looks like there was a failed feed create/update attempt with source type: Third party API and log type: Duo Auth.
Would you like to retry?
Press Enter if you don't want to update.
Enter feed display name[old_display_name]:
(*) Username (Username to authenticate as) [TEEST]
=> TEST
(*) Secret (Secret to authenticate with)
=>
(*) API hostname (The fully qualified domain name for your instance of the API, having the form "api-xxxxxxxx.duosecurity.com") [asd]
=> api-xxxxxxxx.duosecurity.com
Feed updated successfully with Feed ID: 51574667-dee6-408b-a5fc-0e07d3e9a429
delete argument
Use this argument to delete a feed using a feed ID. On execution, it asks for the ID of the feed to be deleted.
Sample usage
$ chronicle_cli feeds delete --help
Usage: chronicle_cli feeds delete [OPTIONS]
Delete a feed
Options:
--url TEXT Base URL to be used for API calls.
--region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
Select region.
-c, --credential-path TEXT Path of Service Account JSON. Default: /usr/
local/google/home/<user>/.chronicle_cli/chronicle_credentials.json
-h, --help Show this message and exit.
Sample output
Enter Feed ID: b0798c54-ed84-44e7-96d5-cbe208f28e49
Feed (ID: b0798c54-ed84-44e7-96d5-cbe208f28e49) deleted successfully.
enable argument
Enables a feed.
Sample usage
$ chronicle_cli feeds enable --help
Usage: main feeds enable [OPTIONS]
Enable feed for the given Feed ID
Options:
--url TEXT Base URL to be used for API calls.
--region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
Select region.
--verbose Prints verbose output to the console.
-c, --credential-path TEXT Path of Service Account JSON. Default: /usr/
local/google/home/<user>/.chronicle_cli/chro
nicle_credentials.json
-h, --help Show this message and exit.
You must enter the feed ID to enable the feed.
Sample output
Enter Feed ID: 29259301-156b-4b60-ae91-855d15c39f6a
Feed with ID: 29259301-156b-4b60-ae91-855d15c39f6a enabled successfully.
disable argument
Disables a feed.
Sample usage
$ chronicle_cli feeds disable --help
Usage: main feeds disable [OPTIONS]
Disable feed for the given Feed ID
Options:
--url TEXT Base URL to be used for API calls.
--region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
Select region.
--verbose Prints verbose output to the console.
-c, --credential-path TEXT Path of Service Account JSON. Default: /usr/
local/google/home/<user>/.chronicle_cli/chro
nicle_credentials.json
-h, --help Show this message and exit.
You must enter the feed ID to disable the feed.
Sample output
Enter Feed ID: 29259301-156b-4b60-ae91-855d15c39f6a
Feed with ID: 29259301-156b-4b60-ae91-855d15c39f6a disabled successfully.
Options
Help (-h / --help)
Use -h or --help option to view usage/description for any command/option.
Sample usage
$ chronicle_cli feeds get -h
Usage: main feeds get [OPTIONS]
Get feed details using Feed ID
Options:
--url TEXT Base URL to be used for API calls.
--region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
Select region.
--verbose Prints verbose output to the console
-c, --credential-path TEXT Path of Service Account JSON. Default: /usr/
local/google/home/<user>/.chronicle_cli/chron
icle_credentials.json
-h, --help Show this message and exit.
Credentials Path (-c or --credential-path)
This option allows to specify the path for the service account
credentials to be used for the authentication. If this option is not specified,
then the Google Security Operations CLI looks for credentials in the default path,
i.e. ~/.chronicle_cli (inside a hidden directory named .chronicle_cli under the home directory).
Sample usage
$ chronicle_cli feeds list --credential-path=C:\chronicle_credentials.json
Verbose (--verbose)
The Google Security Operations CLI prints more details to the console, such as HTTP requests and responses, when this flag is used.
Sample usage
$ chronicle_cli feeds list --verbose
Export (--export)
This option allows you to specify the file path where the output of the list
command will be exported. Both relative and absolute paths are supported.
Sample usage
$ chronicle_cli feeds list --export=$HOME/listFeedsResponse.txt
File Format (--file-format)
This option allows you to specify the file format of the contents exported with
the list command. There are three formats supported: CSV, JSON and TXT. If this
option is not specified with the --export option, then CSV format would be used as default.
Sample usage
$ chronicle_cli feeds list --export=$HOME/listFeedsResponse.txt --file-format=TXT
Sample output
CSV format
ID,Display Name,Source type,Log type,State,Feed Settings
29259301-156b-4b60-ae91-855d15c39f6a,,Third party API,Anomali,INACTIVE,
292b7629-0250-476c-9fb2-4c8a738ce42c,my_duo_auth_feed,Third party API,Duo Auth,ACTIVE,API hostname: api-xxxxxxxxabjdsfklsadlfnsafs.duosecurity.com
0d063a7f-34a1-4dd0-9dcf-9c7a0bb03e65,,Third party API,Workspace Activities,ACTIVE,"Customer ID: C12abc Applications: ['drive', 'login']"
TXT format
Feed Details:
ID: 29259301-156b-4b60-ae91-855d15c39f6a
Source type: Third party API
Log type: Anomali
State: INACTIVE
============================================================
Feed Details:
ID: 292b7629-0250-476c-9fb2-4c8a738ce42c
Display Name: my_duo_auth_feed
Source type: Third party API
Log type: Duo Auth
State: ACTIVE
Feed Settings:
API hostname: api-test.duosecurity.com
============================================================
JSON format
[
{
"name": "feeds/29259301-156b-4b60-ae91-855d15c39f6a",
"details": {
"logType": "ANOMALI_IOC",
"feedSourceType": "API",
"anomaliSettings": {}
},
"feedState": "INACTIVE"
},
{
"name": "feeds/292b7629-0250-476c-9fb2-4c8a738ce42c",
"details": {
"logType": "DUO_AUTH",
"feedSourceType": "API",
"duoAuthSettings": {
"hostname": "api-test.duosecurity.com"
}
},
"feedState": "ACTIVE",
"displayName": "my_duo_auth_feed"
}
]
Region (--region)
You can select the region by passing the --region flag when you run a command.
For more information on how to set the default region, see Set default region.
Troubleshooting
This section shows the output displayed on the console against different types of response codes received from API response.
get argument response codes
| Response Code | Console Output |
| 404 | Invalid feed ID. Please enter valid feed ID. |
| 400 | Feed does not exist. |
| Any other response code | Error while fetching feed.
Response Code: {status code}
Error: {error message}
|
list argument response codes
| Response Code | Console Output |
| Any response code other than 200 | Error while fetching list of feeds.
Response Code: {status code}
Error: {error message}
|
| Any feed in the list failed to fetch | At the end of the console output a list will be printed with the details of the feed ID and corresponding error message. |
create argument response codes
| Response Code | Console Output |
| Any response code other than 200 | Error occurred while creating feed.
Response Code: {status code}
Error: {error message}
|
update argument response codes
| Response Code | Console Output |
| Any response code other than 200 | Error occurred while updating feed. Response Code: {status code}
Error: {error message}
|
delete argument response codes
| Response Code | Console Output |
| 404 | Invalid feed ID. Please enter valid feed ID. |
| 400 | Feed does not exist. |
| Any other response code | Error while deleting feed.
Response Code: {status code}
Error: {error message}
|
enable argument response codes
| Response Code | Console Output |
| 404 | Invalid feed ID. Please enter valid feed ID. |
| 400 | Feed does not exist. |
| Any other response code | Error while enabling feed.
Response Code: {status code}
Error: {error message}
|
disable argument response codes
| Response Code | Console Output |
| 404 | Invalid feed ID. Please enter valid feed ID. |
| 400 | Feed does not exist. |
| Any other response code | Error while disabling feed.
Response Code: {status code}
Error: {error message}
|
Other errors or exceptions
| Exception | Console Output |
| KeyError | Failed to find key {key name} in the response.
|
| Exception | Failed with exception: {exception details}
|
| Credential file missing | Failed with exception: [Errno 2] No such file or directory: '/usr/local/google/home/ You need to place credentials in the expected directory. Refer to Installation. |
Parser management v2 user workflows
You can use Google SecOps CLI to manage configuration-based normalizer (CBN) parsers. We recommend using parser management v2 CLI commands.
Commands
parsers command
The parsers command takes these arguments:
list_parserslist_extensionsrun_parsersubmit_parsersubmit_extensiondelete_parserdelete_extensiondeactivate_parseractivate_parserget_parserget_extensionget_validation_report
Usage syntax:
$ chronicle_cli parsers ARGUMENT [OPTIONS]
Arguments
All CBN parser management workflows in the Google SecOps CLI are interactive. If required, you can also use the command options.
list_parsers argument
Lists all parsers.
$ chronicle_cli parsers list_parsers -h
Usage: chronicle_cli parsers list_parsers [OPTIONS] PROJECT_ID CUSTOMER_ID LOG_TYPE
[New]List all parsers for a given customer
Options:
-s, --state [ALL|ACTIVE|INACTIVE]
Filter on Parser State.
-f, --file-format [TXT|JSON] Format of the file to be exported.
--export TEXT Export output to specified file path.
--env [prod|test] Optionally specify the environment for API
calls.
--region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
Select region.
--verbose Prints verbose output to the console.
-c, --credential_file TEXT Path of Service Account JSON. Default: /usr/
local/home/<user>/.chronicle_cli
/chronicle_credentials.json
--v2 Enable v2 commands.
-h, --help Show this message and exit.
Sample output
Fetching list of parsers... Parser Details: Parser ID: 1242538299340357633 Log type: GCP_CLOUDAUDIT State: INACTIVE Type: CUSTOM Author: - Validation Report ID: 44684d8a-1d01-4e69-ab50-2e2d6e3ef3b2 Create Time: 2023-07-05T05:36:31.121236Z ============================================================ Parser Details: Parser ID: 3840440184193679361 Log type: GCP_CLOUDAUDIT State: INACTIVE Type: CUSTOM Author: - Validation Report ID: 3d2e1bdb-2793-48d1-a485-4f4748095cb8 Create Time: 2023-04-14T09:15:13.718842Z ============================================================ Parser Details: Parser ID: 3651720008402206721 Log type: GCP_SECURITYCENTER_ERROR State: ACTIVE Type: CUSTOM Author: - Validation Report ID: - Create Time: 2023-03-30T09:54:20.414510Z ============================================================
list_extensions argument
Lists all parser extensions.
$ chronicle_cli parsers list_extensions -h
Usage: chronicle_cli parsers list_extensions [OPTIONS] PROJECT_ID CUSTOMER_ID LOG_TYPE
[New]List all extensions for a given customer
Options:
-f, --file-format [TXT|JSON] Format of the file to be exported.
--export TEXT Export output to specified file path.
--env [prod|test] Optionally specify the environment for API
calls.
--region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
Select region.
--verbose Prints verbose output to the console.
-c, --credential_file TEXT Path of Service Account JSON. Default: /usr/
local/home/<user>/.chronicle_cli
/chronicle_credentials.json
--v2 Enable v2 commands.
-h, --help Show this message and exit.
Sample output
Fetching list of Parser Extensions... ParserExtension Details: ParserExtension ID: 7b948bfb-d3f5-4922-9153-a20e75085990 Log type: BRO_DNS State: VALIDATED Validation Report ID: 6ef30ad9-db89-4f30-80f3-0f79758ff3c2 Create Time: 2023-07-06T03:58:26.594863Z State Last Changed Time: 2023-07-06T03:58:26.667151Z Last Live Time: 2023-07-06T03:58:28.019050Z ============================================================ ParserExtension Details: ParserExtension ID: 0fd9129b-d02b-42f7-912a-04b0bba0e0a7 Log type: GCP_DNS State: LIVE Validation Report ID: 1965880f-7cd7-4943-9adf-4bff0041793d Create Time: 2023-05-12T08:12:17.090559Z State Last Changed Time: 2023-05-12T08:12:17.271615Z Last Live Time: 2023-05-12T08:12:27.244342Z ============================================================ ParserExtension Details: ParserExtension ID: d9df9d75-bb3a-4c28-b18d-69a608762ecc Log type: GCP_VPC_FLOW State: REJECTED Validation Report ID: c59ef2ab-4a70-4373-bdc8-067c39ca5a40 Create Time: 2023-04-13T04:43:12.884287Z State Last Changed Time: 2023-04-13T04:43:13.288338Z Last Live Time: - ============================================================
run_parser argument
To validate the parser against given logs, use the following command:
$ chronicle_cli parsers run_parser -h
Usage: chronicle_cli parsers run_parser [OPTIONS] PROJECT_ID CUSTOMER_ID LOG_TYPE
PARSER_CONFIG_FILE LOG_FILE
[New]Run a parser(with extension) against given logs
Options:
--parserextension_config_file TEXT
Path to extension config file.
--env [prod|test] Optionally specify the environment for API
calls.
--region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
Select region.
--verbose Prints verbose output to the console.
-c, --credential_file TEXT Path of Service Account JSON. Default: /usr/
local/home/<user>/.chronicle_cli
/chronicle_credentials.json
--v2 Enable v2 commands.
-h, --help Show this message and exit.
Sample output
Running parser(with extension) against given logs...
{"host_ip": "1.1.1.1"}
{'events': [{'event': {'metadata': {'eventTimestamp': '2023-06-26T08:45:10Z', 'eventType': 'GENERIC_EVENT', 'logType': 'BRO_DNS'}, 'principal': {'ip': ['1.1.1.1']}}}]}
some thing
{}
Runtime: 1.2396s
submit_parser argument
Submits a new parser. The submitted parser goes through validation and the existing parser is promoted to a rollback candidate.
$ chronicle_cli parsers submit_parser -h
Usage: main parsers submit_parser [OPTIONS] PROJECT_ID CUSTOMER_ID LOG_TYPE
CONFIG_FILE [AUTHOR]
[New]Submit a new parser
Options:
--skip_validation_on_no_logs Skip validation if no logs are found.
--env [prod|test] Optionally specify the environment for API
calls.
--region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
Select region.
--verbose Prints verbose output to the console.
-c, --credential_file TEXT Path of Service Account JSON. Default: /usr/
local/home/<user>/.chronicle_cli
/chronicle_credentials.json
--v2 Enable v2 commands.
-h, --help Show this message and exit.
Sample output
Submitting Parser... Parser Details: Parser ID: 12774126091501569 Log type: GCP_CLOUDAUDIT State: INACTIVE Type: CUSTOM Author: - Validation Report ID: - Create Time: 2023-07-06T13:58:10.475391Z ============================================================
submit_extension argument
Submits a new parser extension. The submitted parser extension will go through validation. If validation passes, a new parser extension is created.
$ chronicle_cli parsers submit_extension -h
Usage: chronicle_cli parsers submit_extension [OPTIONS] PROJECT_ID CUSTOMER_ID LOG_TYPE
CONFIG_FILE LOG_FILE
[New]Submit a new extension
Options:
--env [prod|test] Optionally specify the environment for API
calls.
--region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
Select region.
--verbose Prints verbose output to the console.
-c, --credential_file TEXT Path of Service Account JSON. Default: /usr/
local/home/<user>/.chronicle_cli
/chronicle_credentials.json
--v2 Enable v2 commands.
-h, --help Show this message and exit.
Sample output
Submitting Parser Extension... ParserExtension Details: ParserExtension ID: 88907461-c115-4204-8391-425b7a9cfb2c Log type: WORKSPACE_CHROMEOS State: NEW Validation Report ID: - Create Time: 2023-07-06T13:58:10.475391Z State Last Changed Time: - Last Live Time: - ============================================================
delete_parser argument
Deletes a custom parser. You can start using the prebuilt parser for the given log type.
$ chronicle_cli parsers delete_parser -h
Usage: chronicle_cli parsers delete_parser [OPTIONS] PROJECT_ID CUSTOMER_ID LOG_TYPE
PARSER_ID
[New]Delete a parser
Options:
--env [prod|test] Optionally specify the environment for API
calls.
--region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
Select region.
--verbose Prints verbose output to the console.
-c, --credential_file TEXT Path of Service Account JSON. Default: /usr/
local/home/<user>/.chronicle_cli
/chronicle_credentials.json
--v2 Enable v2 commands.
-h, --help Show this message and exit.
Sample output
Deleting Parser… Parser deleted successfully.
delete_extension argument
Deletes a custom parser. You can start using the prebuilt parser for the given log type.
$ chronicle_cli parsers delete_extension -h
Usage: chronicle_cli parsers delete_extension [OPTIONS] PROJECT_ID CUSTOMER_ID LOG_TYPE
PARSEREXTENSION_ID
[New]Delete an extension
Options:
--env [prod|test] Optionally specify the environment for API
calls.
--region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
Select region.
--verbose Prints verbose output to the console.
-c, --credential_file TEXT Path of Service Account JSON. Default: /usr/
local/home/<user>/.chronicle_cli
/chronicle_credentials.json
--v2 Enable v2 commands.
-h, --help Show this message and exit.
Sample output
Deleting Parser Extension… ParserExtension deleted successfully.
deactivate_parser argument
Deactivates an active custom parser. If the deactivation is successful, you can start using the prebuilt parser for the given log type.
$ chronicle_cli parsers deactivate_parser -h
Usage: chronicle_cli parsers deactivate_parser [OPTIONS] PROJECT_ID CUSTOMER_ID
LOG_TYPE PARSER_ID
[New]Deactivate a parser
Options:
--env [prod|test] Optionally specify the environment for API
calls.
--region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
Select region.
--verbose Prints verbose output to the console.
-c, --credential_file TEXT Path of Service Account JSON. Default: /usr/
local/home/<user>/.chronicle_cli
/chronicle_credentials.json
--v2 Enable v2 commands.
-h, --help Show this message and exit.
Sample output
Deactivating Parser… Parser deactivated successfully.
activate_parser argument
Activates a custom parser. You can start using an active parser.
$ chronicle_cli parsers activate_parser -h
Usage: chronicle_cli parsers activate_parser [OPTIONS] PROJECT_ID CUSTOMER_ID LOG_TYPE
PARSER_ID
[New]Activate a parser
Options:
--env [prod|test] Optionally specify the environment for API
calls.
--region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
Select region.
--verbose Prints verbose output to the console.
-c, --credential_file TEXT Path of Service Account JSON. Default: /usr/
local/home/<user>/.chronicle_cli
/chronicle_credentials.json
--v2 Enable v2 commands.
-h, --help Show this message and exit.
Sample output
Activating Parser… Parser activated successfully.
get_parser argument
Fetches the details of the given parser ID and log type.
$ chronicle_cli parsers get_parser -h
Usage: chronicle_cli parsers get_parser [OPTIONS] PROJECT_ID CUSTOMER_ID LOG_TYPE
PARSER_ID
[New]Get details of a parser
Options:
-f, --file-format [TXT|JSON] Format of the file to be exported.
--export TEXT Export output to specified file path.
--env [prod|test] Optionally specify the environment for API
calls.
--region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
Select region.
--verbose Prints verbose output to the console.
-c, --credential_file TEXT Path of Service Account JSON. Default: /usr/
local/home/<user>/.chronicle_cli
/chronicle_credentials.json
--v2 Enable v2 commands.
-h, --help Show this message and exit.
Sample output
Fetching Parser details... Parser Details: Parser ID: 3840440184193679361 Log type: GCP_CLOUDAUDIT State: INACTIVE Type: CUSTOM Author: - Validation Report ID: 3d2e1bdb-2793-48d1-a485-4f4748095cb8 Create Time: 2023-04-14T09:15:13.718842Z ============================================================
get_extension argument
Fetches the configuration of the given parser ID and log type.
$ chronicle_cli parsers get_extension -h
Usage: chronicle_cli parsers get_extension [OPTIONS] PROJECT_ID CUSTOMER_ID LOG_TYPE
PARSEREXTENSION_ID
[New]Get details of an extension
Options:
-f, --file-format [TXT|JSON] Format of the file to be exported.
--export TEXT Export output to specified file path.
--env [prod|test] Optionally specify the environment for API
calls.
--region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
Select region.
--verbose Prints verbose output to the console.
-c, --credential_file TEXT Path of Service Account JSON. Default: /usr/
local/home/<user>/.chronicle_cli
/chronicle_credentials.json
--v2 Enable v2 commands.
-h, --help Show this message and exit.
Sample output
Fetching Parser Extension details... ParserExtension Details: ParserExtension ID: 7b948bfb-d3f5-4922-9153-a20e75085990 Log type: BRO_DNS State: VALIDATED Validation Report ID: 6ef30ad9-db89-4f30-80f3-0f79758ff3c2 Create Time: 2023-07-06T03:58:26.594863Z State Last Changed Time: 2023-07-06T03:58:26.667151Z Last Live Time: 2023-07-06T03:58:28.019050Z ============================================================
get_validation_report argument
Fetches the validation report for a parser or an extension.
$ chronicle_cli parsers get_validation_report [OPTIONS] PROJECT_ID CUSTOMER_ID
LOG_TYPE VALIDATION_REPORT_ID
[New]Get validation report for a parser/extension
Options:
--parser_id TEXT ID of the parser.
--parserextension_id TEXT ID of the parser extension.
--env [prod|test] Optional: Specify the environment for API
calls.
--region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
Select a region.
--verbose Prints verbose output to the console.
-c, --credential_file TEXT Path of Service Account JSON. Default: /usr/
local/google/home/<user>/.chronicle_cli
/chronicle_credentials.json
--v2 Enable v2 commands.
-h, --help Show this message and exit.
Sample output
Fetching Validation report for ParserExtension...
Validation Report:
Verdict: PASS
Stats:
LogEntry Count: 10000
Successfully Normalized Log Count: 10000
Failed Log Count: 0
Invalid Log Count: 0
On Error Count: 153938
Event Count: 10000
Generic Event Count: 0
Event Category:
Valid_event: 10000
Drop Tag:
-
Max Parse Duration: 0.274677769s
Avg Parse Duration: 0.010s
Normalization percent: 100
Generic Event percent: 0
Errors: -
Options
Help (-h / --help)
Use the -h or --help option to view usage with description for any command.
Sample Usage:
$ chronicle_cli parsers list_parsers -h
Usage: chronicle_cli parsers list_parsers [OPTIONS] PROJECT_ID CUSTOMER_ID LOG_TYPE
[New]List all parsers for a given customer
Options:
-s, --state [ALL|ACTIVE|INACTIVE]
Filter on Parser State.
-f, --file-format [TXT|JSON] Format of the file to be exported.
--export TEXT Export output to specified file path.
--env [prod|test] Optionally specify the environment for API
calls.
--region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
Select region.
--verbose Prints verbose output to the console.
-c, --credential_file TEXT Path of Service Account JSON. Default: /usr/
local/home/<user>/.chronicle_cli
/chronicle_credentials.json
--v2 Enable v2 commands.
-h, --help Show this message and exit.
Environment (--env)
You can select the environment by passing the --env flag with command and the API calls will be executed accordingly.
You can select an env value from the following: prod, test.
If you do not specify this option, the default is set to prod.
Credentials File (-c or --credential_file)
This option allows to specify the path for the service account
credentials to be used for authentication.
If this option is not specified, then the Google SecOps CLI looks for credentials in
the default path, i.e. ~/.chronicle_cli (inside a hidden directory named .chronicle_cli under the home directory).
Sample Usage:
$ chronicle_cli parsers list_parsers --credential_file=C:\chronicle_credentials.json
Region (--region)
You can select the region by passing the --region flag when you run a command.
For more information on how to set the default region, see Set default region.
Verbose (--verbose)
This option allows you to print the details of the HTTP Request made and Response received.
Sample Usage:
$ chronicle_cli parsers list_parsers --verbose
Export (--export)
This option allows you to specify the file path where the output of the list or list_errors command will be exported. Both relative and absolute paths are supported.
Sample Usage:
$ chronicle_cli parsers list_parsers --export=parser_list
File Format (--file-format)
This option allows you to specify the file format of the contents exported with the list or list_errors command. Two formats are supported: JSON and TXT. If this option is not specified with the --export option, then TXT format is used as default.
Sample Usage:
$ chronicle_cli parsers list_parsers --export=parser_list --file-format=JSON
Parser management user workflows
The Google SecOps CLI can be used for managing CBN parsers using the following commands. However, it is recommended to use parser management v2 CLI commands.
Commands
parsers command
The parsers command takes these arguments:
archivedownloadgeneratehistorylistlist_errorsrunstatussubmit
Usage syntax:
$ chronicle_cli parsers ARGUMENT [OPTIONS]
Arguments
All CBN parser management workflows in the Google SecOps CLI are interactive. You are prompted for options when they are required for commands.
list argument
To list the details of all parsers, use the following command:
$ chronicle_cli parsers list -h
Usage: main parsers list [OPTIONS]
List all parsers of a given customer
Options:
--env [prod|test] Optionally specify the environment for API
calls.
--region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
Select region.
--export TEXT Export output to specified file path.
--file-format [TXT|CSV|JSON] Format of the file to be exported.
--verbose Prints verbose output to the console.
-c, --credential_file TEXT Path of Service Account JSON. Default: /usr/
local/home/<user>/.chronicle_cli/chro
nicle_credentials.json
-h, --help Show this message and exit.
Sample output
Fetching list of parsers...
Parser Details:
Config ID: 1cb402d9-eab2-4f6b-b402-20b1211675ed
Log type: WINDOWS_SYSMON
State: LIVE
SHA256: 7xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx7
Author: <user>@test.com
Submit Time: 2022-08-26T09:57:10.644351Z
State Last Changed Time: 2022-08-26T09:58:23.809636Z
Last Live Time: 2022-08-26T09:58:23.809636Z
============================================================
Parser Details:
Config ID: 7f2ae1f5-8f0c-43f9-bb02-299e7c8b9e82
Log type: BOX
State: LIVE
SHA256: 8xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx7
Author: <user>@test.com
Submit Time: 2022-08-25T07:33:31.026399Z
State Last Changed Time: 2022-08-25T07:33:32.263754Z
Last Live Time: 2022-08-25T07:33:32.263754Z
============================================================
generate argument
To generate sample logs for a given log type, use the following command:
$ chronicle_cli parsers generate -h
Usage: main parsers generate [OPTIONS]
Generate sample logs for a given log type
Options:
--env [prod|test] Optionally specify the environment for API
calls.
--region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
Select region.
-c, --credential_file TEXT Path of Service Account JSON. Default: /usr/
local/home/<user>/.chronicle_cli/chro
nicle_credentials.json
-h, --help Show this message and exit.
The command creates three files with 1, 10, and 1,000 sample logs at the root
directory under <root>/chronicle_cli/cbn/<log_type>/.
Sample output
Enter Start Date (Format: yyyy-mm-ddThh:mm:ssZ): 2022-08-17T10:00:00Z
Enter End Date (Format: yyyy-mm-ddThh:mm:ssZ): 2022-08-23T10:00:00Z
Enter Log Type: WINDOWS_DHCP
Generating sample size: 1...
Generating sample size: 10...
Generating sample size: 1k...
Generated sample data (WINDOWS_DHCP); run this to go there:
cd /usr/local/home/<user>/cbn/windows_dhcp
history argument
To get the list of all parsers submission details for a given log type, use the following command:
$ chronicle_cli parsers history -h
Usage: main parsers history [OPTIONS]
History retrieves all parsers submissions given a log type
Options:
--env [prod|test] Optionally specify the environment for API
calls.
--region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
Select region.
--verbose Prints verbose output to the console.
-c, --credential_file TEXT Path of Service Account JSON. Default: /usr/
local/home/<user>/.chronicle_cli/chro
nicle_credentials.json
-h, --help Show this message and exit.
Sample output
Enter Log Type: WINDOWS_SYSMON
Fetching history for parser...
Parser History:
Config ID: 8d9f5b1c-4689-4ca3-ae9b-863ce78dd123
Log type: WINDOWS_SYSMON
State: LIVE
SHA256: 7xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx7
Author: author@test.com
Submit Time: 2022-08-26T12:37:55.187407Z
State Last Changed Time: 2022-08-26T12:39:12.198587Z
Last Live Time: 2022-08-26T12:39:12.198587Z
============================================================
Parser History:
Config ID: 29bbf14b-2ffb-411a-bb37-911b13437123
Log type: WINDOWS_SYSMON
State: ARCHIVED
SHA256: 8xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx7
Author: author@test.com
Submit Time: 2022-08-26T12:05:34.421743Z
State Last Changed Time: 2022-08-26T12:39:12.198587Z
Last Live Time: 2022-08-26T12:06:55.495269Z
============================================================
list_errors argument
To list errors of a log type between specific timestamps, use the following command:
$ chronicle_cli parsers list_errors -h
Usage: main parsers list_errors [OPTIONS]
List errors of a log type between specific timestamps
Options:
--env [prod|test] Optionally specify the environment for API
calls.
--region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
Select region.
--export TEXT Export output to specified file path.
--file-format [TXT|CSV|JSON] Format of the file to be exported.
--verbose Prints verbose output to the console.
-c, --credential_file TEXT Path of Service Account JSON. Default: /usr/
local/home/<user>/.chronicle_cli/chro
nicle_credentials.json
-h, --help Show this message and exit.
Sample output
Enter Log Type: CISCO_ASA_FIREWALL
Enter Start Date (Format: yyyy-mm-ddThh:mm:ssZ): 2021-01-16T00:00:00Z
Enter End Date (Format: yyyy-mm-ddThh:mm:ssZ): 2022-08-21T12:00:00Z
Getting parser errors...
Error Details:
Error ID: f9eb72cb-f320-dd5a-a098-00bcaa76a35d
Config ID: N/A
Log type: CISCO_ASA_FIREWALL
Error Time: 2022-08-18T10:57:56.898883208Z
Error Category: CBN_parsers_GENERATED_INVALID_EVENT
Error Message: generic::invalid_argument: diff event timestamp ("seconds:1630106465") and create timestamp ("seconds:1660820265 nanos:202151000"): 8531h36m40.202151s, larger than allowed (4320h0m0s)
Logs:
<190>Aug 27 2020 23:21:05 TEST : %ASA-6-106012: Deny IP from 1.2.3.4 to 5.6.7.8, IP options: Test user
============================================================
Error Details:
Error ID: f9eb72cb-f320-dd5a-a098-00bcaa76a35d
Config ID: N/A
Log type: CISCO_ASA_FIREWALL
Error Time: 2022-08-18T10:57:56.898883208Z
Error Category: CBN_parsers_GENERATED_INVALID_EVENT
Error Message: generic::invalid_argument: diff event timestamp ("seconds:1630106465") and create timestamp ("seconds:1660820265 nanos:202151000"): 8531h36m40.202151s, larger than allowed (4320h0m0s)
Logs:
<190>Aug 27 2020 23:21:05 TEST : %ASA-6-106012: Deny IP from 1.2.3.4 to 5.6.7.8, IP options: Demo user
run argument
To validate the parser against given logs, use the following command:
$ chronicle_cli parsers run -h
Usage: main parsers run [OPTIONS]
Run the parser against given logs
Options:
--env [prod|test] Optionally specify the environment for API
calls.
--region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
Select region.
--verbose Prints verbose output to the console.
-c, --credential_file TEXT Path of Service Account JSON. Default: /usr/
local/home/<user>/.chronicle_cli/chro
nicle_credentials.json
-h, --help Show this message and exit.
Sample output
Enter path for conf file: /usr/local/home/Desktop/windows_sysmon.conf
Enter path for log file: /usr/local/home/Desktop/windows_sysmon.log
Running Validation…
Runtime: 2.4914s
submit argument
To submit a new parser, use the following command:
$ chronicle_cli parsers submit -h
Usage: main parsers submit [OPTIONS]
Submit new parser
Options:
--env [prod|test] Optionally specify the environment for API
calls.
--region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
Select region.
--verbose Prints verbose output to the console.
-c, --credential_file TEXT Path of Service Account JSON. Default: /usr/
local/home/<user>/.chronicle_cli/chro
nicle_credentials.json
-h, --help Show this message and exit.
Sample output
Enter Log type: CISCO_ASA_FIREWALL
Enter Config file path: /usr/local/Desktop/windows_sysmon.conf
Enter author: test
Submitting parser...
Submitted Parser Details:
Config ID: 9ba20930-9733-4fcd-badf-18fedb9f8123
Log type: CISCO_ASA_FIREWALL
State: NEW
SHA256: 7xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx7
Author: test
Submit Time: 2022-08-30T06:49:54.005119Z
State Last Changed Time: 2022-08-30T06:49:54.005119Z
Parser submitted successfully. To get status of the parser, run this command using following Config ID - 9ba20930-9733-4fcd-badf-18fedb9f8123:
chronicle_cli parsers status
status argument
To get the status of a submitted parser, use the following command:
$ chronicle_cli parsers status -h
Usage: main parsers status [OPTIONS]
Get status of a submitted parser
Options:
--env [prod|test] Optionally specify the environment for API
calls.
--region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
Select region.
--verbose Prints verbose output to the console.
-c, --credential_file TEXT Path of Service Account JSON. Default: /usr/
local/home/<user>/.chronicle_cli/chro
nicle_credentials.json
-h, --help Show this message and exit.
Sample output
Enter Config ID: 1cb402d9-eab2-4f6b-b402-20b1211675ed
Getting parser...
Parser Details:
Config ID: 1cb402d9-eab2-4f6b-b402-20b1211675ed
Log type: WINDOWS_SYSMON
State: ARCHIVED
SHA256: 79ac67c15ffb047a152be2fb2a3391cbe18b2d183e9e6a402eb2fe53a6666b17
Author: test
Submit Time: 2022-08-26T09:57:10.644351Z
State Last Changed Time: 2022-08-26T09:58:23.809636Z
Last Live Time: 2022-08-26T09:58:23.809636Z
archive argument
To archive an existing parser, use the following command:
$ chronicle_cli parsers archive -h
Usage: main parsers archive [OPTIONS]
Archives a parser given the config ID.
Options:
--env [prod|test] Optionally specify the environment for API
calls.
--region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
Select region.
--verbose Prints verbose output to the console.
-c, --credential_file TEXT Path of Service Account JSON. Default: /usr/
local/home/<user>/.chronicle_cli/chro
nicle_credentials.json
-h, --help Show this message and exit.
Sample output
Enter Config ID: 1cb402d9-eab2-4f6b-b402-20b121167123
Archiving parser...
Parser archived Successfully.
Parser Details:
Config ID: 1cb402d9-eab2-4f6b-b402-20b121167123
Log type: WINDOWS_SYSMON
State: ARCHIVED
SHA256: 7xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx7
Author: test
Submit Time: 2022-08-26T09:57:10.644351Z
State Last Changed Time: 2022-08-26T09:58:23.809636Z
Last Live Time: 2022-08-26T09:58:23.809636Z
download argument
To download the configuration (.conf) file for a given log type or config ID,
use the following command:
$ chronicle_cli parsers download -h
Usage: main parsers download [OPTIONS]
Download parser code by given Config ID or Log type.
Options:
--env [prod|test] Optionally specify the environment for API
calls.
--region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
Select region.
--verbose Prints verbose output to the console.
-c, --credential_file TEXT Path of Service Account JSON. Default: /usr/
local/home/<user>/.chronicle_cli/chro
nicle_credentials.json
-h, --help Show this message and exit.
Sample output
- Using Config ID.
Note: If you want to download parser by log type then skip the config ID.
Enter config ID: 9d1474ab-eff2-4855-ba57-4f0c458e3ac2
Downloading parser...
Writing parser to: CISCO_ASA_FIREWALL_20220825131911.conf
- Using Log Type.
Note: If you want to download parser by log type then skip the config ID.
Enter config ID:
Enter Log Type: CISCO_ASA_FIREWALL
Downloading parser...
Writing parser to: CISCO_ASA_FIREWALL_20220825132011.conf
Options
Help (-h / --help)
Use the -h or --help option to view usage with description for any command.
Sample Usage:
$ chronicle_cli parsers list -h
Usage: main parsers list [OPTIONS]
List all parsers of a given customer
Options:
-f, --file-format [TXT|JSON] Format of the file to be exported.
--export TEXT Export output to specified file path.
--env [prod|test] Optionally specify the environment for API
calls.
--region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
Select region.
--verbose Prints verbose output to the console.
-c, --credential_file TEXT Path of Service Account JSON. Default: /usr/
local/home/<user>/.chronicle_cli/chro
nicle_credentials.json
-h, --help Show this message and exit.
Environment (--env)
You can select the environment by passing the --env flag with command and the API calls will be executed accordingly.
You can select an env value from the following: prod, test.
If you don't specify this option, the default is set to prod.
Credentials File (-c or --credential_file)
This option allows to specify the path for the service account credentials to be used for the authentication of the user. If this option is not specified, then the Google SecOps CLI looks for credentials in the default path, i.e. ~/.chronicle_cli (inside a hidden directory named .chronicle_cli under the home directory).
Sample Usage:
$ chronicle_cli parsers list --credential_file=C:\chronicle_credentials.json
Region (--region)
You can select the region by passing the --region flag when you run a command.
For more information on how to set the default region, see Set default region.
Verbose (--verbose)
This option allows you to print the details of the HTTP Request made and Response received.
Sample Usage:
$ chronicle_cli parsers list --verbose
Export (--export)
This option allows you to specify the file path where the output of the list or
list_errors command will be exported. Both relative and absolute paths are supported.
Sample Usage:
$ chronicle_cli parsers list --export=parsers_list
File Format (--file-format)
This option allows you to specify the file format of the contents exported with the
list or list_errors command. There are three formats supported: JSON and TXT.
If this option is not specified with the --export option, then TXT format would be used as default.
Sample Usage:
$ chronicle_cli parsers list --export=parsers_list --file-format=JSON
Troubleshooting
CBN Parser Management Error Codes
This section shows the output displayed on the console against different types of response code received from API response.
Refer to below table to view output for each command:
| Response Code | Command | Console Output |
| Any response code other than 200 | {command}[archive, download, generate, history, list_errors, run, status, list, submit] |
Error while {command} parser.Response Code: {status code}
Error: {error message} |
Forwarder management user workflows
The Google SecOps CLI can be used to manage forwarders and their associated collectors using the following commands:
Commands
forwarders command
The forwarders command takes these arguments:
createupdategetlistdeletegenerate_filescollectors
Usage syntax:
$ chronicle_cli forwarders ARGUMENT [OPTIONS]
collectors command
The collectors command takes these arguments:
createupdategetlistdelete
Usage syntax:
$ chronicle_cli forwarders collectors ARGUMENT [OPTIONS]
Arguments
All Forwarder Management user workflows in the Google SecOps CLI are interactive. You are prompted for options as and when required.
create argument
To create a new forwarder and configure collectors for it, use the following command:
$ chronicle_cli forwarders create --help
Usage: main forwarders create [OPTIONS]
Create a Forwarder
Options:
--url TEXT Base URL to be used for API calls.
--region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
Select region.
--verbose Prints verbose output to the console.
-c, --credential_file TEXT Path of Service Account JSON. Default: /usr/
local/google/home/<user>/.chronicle_cl
i/chronicle_credentials.json
-h, --help Show this message and exit.
Sample output
$ chronicle_cli forwarders create
================================================================================
Press Enter if you want to use the default value mentioned besides field description in [] brackets.
================================================================================
(*) Forwarder Display Name : test_display_name
========================================
======== Forwarder Configuration =======
========================================
Upload Compression (Determines if uploaded data will be compressed) [Y/n]: y
Do you want to proceed with Forwarder Metadata? [y/N]: y
========================================
========== Forwarder Metadata ==========
========================================
Asset Namespace: test_namespace
========================================
=========== Forwarder Labels ===========
========================================
Labels (The ingestion metadata labels in 'key:value' format to apply to all logs ingested through this forwarder, as well as the resulting normalized data.)
Enter/Paste your content. On a new line, press Ctrl-D (Linux) / [Ctrl-Z + Enter (Windows)] to save it:
key1:value1
key2:value2
Do you want to proceed with Forwarder Regex Filters? [y/N]: y
========================================
======= Forwarder Regex Filters =======
========================================
Filter Description (Describes what is being filtered and why): desc1
Filter Regexp (The regular expression used to match against each incoming line): .*
Filter Behavior (Filter behavior to apply when a match is found)
Choose:
1. allow
2. block
: 1
You have selected allow
Do you want to add more Forwarder Regex Filters [y/N]: y
Filter Description (Describes what is being filtered and why): desc2
Filter Regexp (The regular expression used to match against each incoming line): .*
Filter Behavior (Filter behavior to apply when a match is found)
Choose:
1. allow
2. block
: 2
You have selected block
Do you want to add more Forwarder Regex Filters [y/N]: n
Do you want to proceed with Server Settings? [y/N]: y
========================================
=========== Server Settings ===========
========================================
Server State (Server State for Collector)
Choose:
1. active
2. suspended
: 1
You have selected active
Graceful Timeout (Number of seconds after which the forwarder returns a bad readiness/health check and still accepts new connections) [15]:
Drain timeout (Number of seconds after which the forwarder waits for active connections to successfully close on their own before being closed by the server) [10]:
Do you want to proceed with HTTP-specific server settings? [y/N]: y
========================================
==== HTTP-specific server settings ====
========================================
Host (IP address, or hostname that can be resolved to IP addresses, that the server should listen on) [0.0.0.0]: 10.0.14.132
Port (Port number that the HTTP server listens on for health checks from the load balancer) [8080]: 8000
Read Timeout (Maximum amount of time allowed to read the entire request, both the header and the body) [3]:
Read Header Timeout (Maximum amount of time allowed to read request headers) [3]:
Write Timeout (Maximum amount of time allowed to send a response) [3]:
Idle Timeout (Maximum amount of time (in seconds) to wait for the next request when idle connections are enabled) [3]:
Do you want to proceed with Route Settings? [y/N]: y
========================================
============ Route Settings ============
========================================
Available Status Code (Status code returned when a liveness check is received and the forwarder is available) [204]: 200
Ready Status Code (Status code returned when it is ready to accept traffic) [204]: 200
Unready Status Code (Status code returned when it is not ready to accept traffic) [503]: 500
Preview changes:
- Press Up/b or Down/z keys to paginate.
- To switch case-sensitivity, press '-i' and press enter. By default, search
is case-sensitive.
- To search for specific field, press '/' key, enter text and press enter.
- Press 'q' to quit and confirm preview changes.
- Press `h` for all the available options to navigate the list.
=============================================================================
Config:
Metadata:
Asset namespace: test_namespace
Labels:
- key: key1
value: value1
- key: key2
value: value2
Regex filters:
- behavior: ALLOW
description: desc1
regexp: .*
- behavior: BLOCK
description: desc2
regexp: .*
Server settings:
Drain timeout: 10
Graceful timeout: 15
Http settings:
Host: 10.0.14.132
Idle timeout: 3
Port: 8000
Read header timeout: 3
Read timeout: 3
Do you want to create forwarder with this configuration [y/N]: y
Creating forwarder...
Forwarder created successfully with Forwarder ID: a7e59660-959b-44e7-aa7e-baec820d01f4
Would you like to configure collectors for this forwarder? [y/N]: y
(*) Collector Display Name: collector_1
========================================
======== Collector Configuration ======
========================================
(*) Collector Log Type (Type of logs collected): WINDOWS_DNS
Do you want to proceed with Collector Metadata? [y/N]: y
========================================
========== Collector Metadata ==========
========================================
Asset Namespace: test_namespace
========================================
=========== Forwarder Labels ===========
========================================
Labels (The ingestion metadata labels in 'key:value' format to apply to all logs ingested through this forwarder, as well as the resulting normalized data.)
Enter/Paste your content. On a new line, press Ctrl-D (Linux) / [Ctrl-Z + Enter (Windows)] to save it:
key1:value1
key2:value2
Do you want to proceed with Collector Regex Filters? [y/N]: y
========================================
======= Collector Regex Filters =======
========================================
Filter Description (Describes what is being filtered and why): desc1
Filter Regexp (The regular expression used to match against each incoming line): .*
Filter Behavior (Filter behavior to apply when a match is found)
Choose:
1. allow
2. block
: 1
You have selected allow
Do you want to add more Collector Regex Filters [y/N]: n
Do you want to proceed with Collector Disk Buffer? [y/N]: y
========================================
======== Collector Disk Buffer ========
========================================
Disk Buffer State (Disk buffering state for collector)
Choose:
1. active
2. suspended
: 1
You have selected active
Directory Path (Directory path for files written): path/to/file.txt
Max File Buffer Bytes (Maximum buffered file size): 45
Maximum Seconds per Batch (Maximum number of seconds between forwarder batch uploads) [10]:
Maximum Bytes per Batch (Maximum number of bytes queued before forwarder batch upload) [1048576]:
========================================
===== Configure Ingestion Settings =====
========================================
Choose:
1. File Settings
2. Kafka Settings
3. Pcap Settings
4. Splunk Settings
5. Syslog Settings
: 1
File Path (Path of file to monitor): path/to/file.txt
Preview changes:
- Press Up/b or Down/z keys to paginate.
- To switch case-sensitivity, press '-i' and press enter. By default, search
is case-sensitive.
- To search for specific field, press '/' key, enter text and press enter.
- Press 'q' to quit and confirm preview changes.
- Press `h` for all the available options to navigate the list.
=============================================================================
Config:
Disk buffer:
Directory path: path/to/file.txt
Max file buffer bytes: 45
State: ACTIVE
File settings:
File path: path/to/file.txt
Log type: WINDOWS_DNS
Max bytes per batch: 1048576
Max seconds per batch: 10
Metadata:
Asset namespace: test_namespace
Labels:
- key: key1
value: value1
- key: key2
value: value2
Regex filters:
Behavior: ALLOW
Description: desc1
Regexp: .*
Display name: collector_1
Do you want to create collector with this configuration [y/N]: y
Creating collector...
Collector created successfully with Collector ID: 1f72f9ab-3ae3-4c5f-955e-86c982587937
Would you like to add more collectors? [y/N]: n
If the forwarder create fails, and you enter the same forwarder ID, then you are prompted to retry the failed forwarder or restart the process. If the forwarder ID entered by you does not match with the failed forwarder ID, then you will not be prompted to retry and the normal process of creating the forwarder continues.
Sample output
...
Creating forwarder...
Error occurred while creating forwarder.
Response Code: 500.
Error: ZERO_APP::1: create forwarder due to validation errors in request: generic::invalid_argument: filter's description is not specified
$ chronicle_cli forwarders create
Looks like there was a failed create/update attempt for test.
Would you like to retry?
(*) Forwarder Display Name [test]:
Do you want to create forwarder with this configuration [y/N]: y
Creating forwarder...
Forwarder created successfully with Forwarder ID: ab7af569-d957-44a3-99a8-aa70ffdc6458
Would you like to configure collectors for this forwarder? [y/N]: n
get argument
To get details of an existing forwarder and respective collectors, use the following command:
$ chronicle_cli forwarders get --help
Usage: main forwarders get [OPTIONS]
Get forwarder details using Forwarder ID
Options:
--url TEXT Base URL to be used for API calls.
--region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
Select region.
--verbose Prints verbose output to the console.
-c, --credential_file TEXT Path of Service Account JSON. Default: /usr/
local/google/home/<user>/.chronicle_cl
i/chronicle_credentials.json
-h, --help Show this message and exit.
Sample output
You must enter the forwarder ID to get the forwarder details.
$ chronicle_cli forwarders get
Enter Forwarder ID: a7e59660-959b-44e7-aa7e-baec820d01f4
Fetching forwarder and its all associated collectors...
Forwarder Details:
ID: a7e59660-959b-44e7-aa7e-baec820d01f4
Display name: test_display_name
State: ACTIVE
Config:
Upload compression: true
Metadata:
Asset namespace: test_namespace
Labels:
- key: key1
value: value1
- key: key2
value: value2
Regex filters:
- description: desc1
regexp: .*
behavior: ALLOW
- description: desc2
regexp: .*
behavior: BLOCK
Server settings:
Graceful timeout: 15
Drain timeout: 10
Http settings:
Port: 8000
Host: 10.0.14.132
Read timeout: 3
Read header timeout: 3
Write timeout: 3
Idle timeout: 3
Route settings:
Available status code: 200
Ready status code: 200
Unready status code: 500
State: ACTIVE
Collectors:
Collector [1f72f9ab-3ae3-4c5f-955e-86c982587937]:
Display name: collector_1
State: ACTIVE
Config:
Log type: WINDOWS_DNS
Metadata:
Asset namespace: test_namespace
Labels:
- key: key1
value: value1
- key: key2
value: value2
Regex filters:
- description: desc1
regexp: .*
behavior: ALLOW
Disk buffer:
State: ACTIVE
Directory path: path/to/file.txt
Max file buffer bytes: '45'
Max seconds per batch: 10
Max bytes per batch: '1048576'
File settings:
File path: path/to/file.txt
================================================================================
list argument
To display list of all forwarders and respective collectors, use following command:
$ chronicle_cli forwarders list --help
Usage: main forwarders list [OPTIONS]
List all forwarders
Options:
--url TEXT Base URL to be used for API calls.
--region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
Select region.
--verbose Prints verbose output to the console.
--export TEXT Export output to specified file path
--file-format [TXT|CSV|JSON] Format of the file to be exported
-c, --credential_file TEXT Path of Service Account JSON. Default: /usr/
local/google/home/<user>/.chronicle_cl
i/chronicle_credentials.json
-h, --help Show this message and exit.
Command is used to fetch the details of all the forwarders and collectors.
Sample output
$ chronicle_cli forwarders list
Fetching list of forwarders...
Forwarder Details:
ID: a7e59660-959b-44e7-aa7e-baec820d01f4
Display name: test_display_name
State: ACTIVE
Config:
Upload compression: true
Metadata:
Asset namespace: test_namespace
Labels:
- key: key1
value: value1
- key: key2
value: value2
Regex filters:
- description: desc1
regexp: .*
behavior: ALLOW
- description: desc2
regexp: .*
behavior: BLOCK
Server settings:
Graceful timeout: 15
Drain timeout: 10
Http settings:
Port: 8000
Host: 10.0.14.132
Read timeout: 3
Read header timeout: 3
Write timeout: 3
Idle timeout: 3
Route settings:
Available status code: 200
Ready status code: 200
Unready status code: 500
State: ACTIVE
Collectors:
Collector [1f72f9ab-3ae3-4c5f-955e-86c982587937]:
Display name: collector_1
State: ACTIVE
Config:
Log type: WINDOWS_DNS
Metadata:
Asset namespace: test_namespace
Labels:
- key: key1
value: value1
- key: key2
value: value2
Regex filters:
- description: desc1
regexp: .*
behavior: ALLOW
Disk buffer:
State: ACTIVE
Directory path: path/to/file.txt
Max file buffer bytes: '45'
Max seconds per batch: 10
Max bytes per batch: '1048576'
File settings:
File path: path/to/file.txt
================================================================================
Forwarder Details:
ID: ddcca884-cdc6-4ac2-ad30-05a28e6cf35a
Display name: test
State: ACTIVE
Config:
Upload compression: true
Metadata:
Asset namespace: test
Labels:
- key: k1
value: v2
Regex filters:
- description: hh
regexp: hh
behavior: ALLOW
- description: gg
regexp: gg
behavior: BLOCK
Server settings:
Graceful timeout: 15
Drain timeout: 10
Http settings:
Port: 8080
Host: 0.0.0.0
Read timeout: 3
Read header timeout: 3
Write timeout: 3
Idle timeout: 3
Route settings:
Available status code: 204
Ready status code: 204
Unready status code: 503
State: ACTIVE
Collectors:
Message: No collectors found for this forwarder.
================================================================================
If you want to export the data, then you can specify the absolute/relative path of the file to be exported along with the format of the file (CSV/TXT/JSON). The default file format is CSV.
Sample output
$ chronicle_cli forwarders list --export=$HOME/listforwarder --file-format=JSON
Fetching list of forwarders...
Forwarder Details:
ID: a7e59660-959b-44e7-aa7e-baec820d01f4
Display name: test_display_name
State: ACTIVE
Config:
Upload compression: true
Metadata:
Asset namespace: test_namespace
Labels:
- key: key1
value: value1
- key: key2
value: value2
Regex filters:
- description: desc1
regexp: .*
behavior: ALLOW
- description: desc2
regexp: .*
behavior: BLOCK
Server settings:
Graceful timeout: 15
Drain timeout: 10
Http settings:
Port: 8000
Host: 10.0.14.132
Read timeout: 3
Read header timeout: 3
Write timeout: 3
Idle timeout: 3
Route settings:
Available status code: 200
Ready status code: 200
Unready status code: 500
State: ACTIVE
Collectors:
Collector [1f72f9ab-3ae3-4c5f-955e-86c982587937]:
Display name: collector_1
State: ACTIVE
Config:
Log type: WINDOWS_DNS
Metadata:
Asset namespace: test_namespace
Labels:
- key: key1
value: value1
- key: key2
value: value2
Regex filters:
- description: desc1
regexp: .*
behavior: ALLOW
Disk buffer:
State: ACTIVE
Directory path: path/to/file.txt
Max file buffer bytes: '45'
Max seconds per batch: 10
Max bytes per batch: '1048576'
File settings:
File path: path/to/file.txt
================================================================================
Forwarder Details:
ID: ddcca884-cdc6-4ac2-ad30-05a28e6cf35a
Display name: test
State: ACTIVE
Config:
Upload compression: true
Metadata:
Asset namespace: test
Labels:
- key: k1
value: v2
Regex filters:
- description: hh
regexp: hh
behavior: ALLOW
- description: gg
regexp: gg
behavior: BLOCK
Server settings:
Graceful timeout: 15
Drain timeout: 10
Http settings:
Port: 8080
Host: 0.0.0.0
Read timeout: 3
Read header timeout: 3
Write timeout: 3
Idle timeout: 3
Route settings:
Available status code: 204
Ready status code: 204
Unready status code: 503
State: ACTIVE
Collectors:
Message: No collectors found for this forwarder.
================================================================================
Forwarders list details exported successfully to: /usr/local/google/home/<user>/listforwarder.json
update argument
To update an existing forwarder, use the following command:
$ chronicle_cli forwarders update --help
Usage: main forwarders update [OPTIONS]
Update a forwarder using forwarder ID.
Options:
--url TEXT Base URL to be used for API calls.
--region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
Select region.
--verbose Prints verbose output to the console.
-c, --credential_file TEXT Path of Service Account JSON. Default: /usr/
local/google/home/<user>/.chronicle_cl
i/chronicle_credentials.json
-h, --help Show this message and exit.
After running the command, enter the Forwarder ID and all the field values again. You can reuse the old values by pressing the Enter.
Sample output
$ chronicle_cli forwarders update
Enter Forwarder ID: a7e59660-959b-44e7-aa7e-baec820d01f4
Press Enter if you don't want to update.
(*) Forwarder Display Name [test_display_name]:
========================================
======== Forwarder Configuration =======
========================================
Upload Compression (Determines if uploaded data will be compressed) [Y/n]: y
Do you want to proceed with Forwarder Metadata? [y/N]: y
========================================
========== Forwarder Metadata ==========
========================================
Asset Namespace [test_namespace]:
========================================
=========== Forwarder Labels ===========
========================================
Labels (The ingestion metadata labels in 'key:value' format to apply to all logs ingested through this forwarder, as well as the resulting normalized data.)
Enter/Paste your content. On a new line, press Ctrl-D (Linux) / [Ctrl-Z + Enter (Windows)] to save it:
[[{'key': 'key1', 'value': 'value1'}, {'key': 'key2', 'value': 'value2'}]]
Do you want to proceed with Forwarder Regex Filters? [y/N]: n
Do you want to proceed with Server Settings? [y/N]: n
Do you want to update forwarder with this configuration? [y/N]: y
Updating forwarder...
Forwarder updated successfully with Forwarder ID: a7e59660-959b-44e7-aa7e-baec820d01f4
If the forwarder update fails and you enter the same forwarder, then a prompt is displayed to retry the failed forwarder or to restart the process. If you enter a forwarder ID which does not match the failed forwarder ID, then the option to retry is not displayed and the normal process of updating the forwarder continues. The retry mechanism allows you to change option values provided in the previous failed attempt in an interactive way. Press Enter to reuse the same value for an option in the forwarder update flow.
delete argument
Use this argument to delete a forwarder using a forwarder ID. On execution, it asks for the ID of the feed to be deleted. To delete an existing forwarder, use the following command:
chronicle_cli forwarders delete --help
Usage: main forwarders delete [OPTIONS]
Delete a forwarder using Forwarder ID
Options:
--url TEXT Base URL to be used for API calls.
--region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
Select region.
--verbose Prints verbose output to the console.
-c, --credential_file TEXT Path of Service Account JSON. Default: /usr/
local/google/home/<user>/.chronicle_cl
i/chronicle_credentials.json
-h, --help Show this message and exit.
Sample output
$ chronicle_cli forwarders delete
Enter Forwarder ID: 0593ba21-a1c7-4279-b429-bc8df959bd59
Deleting forwarder and all its associated collectors...
Forwarder (ID: 0593ba21-a1c7-4279-b429-bc8df959bd59) deleted successfully with all its associated collectors.
generate_files argument
Use this argument to generate file having information about forwarder using forwarder ID.
To generate the forwarder file, use the following command:
$ chronicle_cli forwarders generate_files -h
Usage: main forwarders generate_files [OPTIONS]
Generate forwarder configuration files using Forwarder ID
Options:
--url TEXT Base URL to be used for API calls.
--region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
Select region.
--verbose Prints verbose output to the console.
-c, --credential_file TEXT Path of Service Account JSON. Default: /usr/
local/google/home/<user>/.chronicle_cl
i/chronicle_credentials.json
-f, --file-path TEXT Download generated forwarder files to the
specified path.
-h, --help Show this message and exit.
Sample output
$ chronicle_cli forwarders generate_files --file-path=$HOME/GenerateForwarderFile
Enter Forwarder ID: 0768220e-8af6-4ef7-a1dd-73e33963b444
Generating forwarder files ...
Forwarder files generated successfully.
Configuration file: /usr/local/google/home/<user>/GenerateForwarderFile_forwarder.conf
Auth file: /usr/local/google/home/<user>/GenerateForwarderFile_forwarder_auth.conf
Collector sub commands
create argument
Use the following command to configure a new collector for a given forwarder.
$ chronicle_cli forwarders collectors create --help
Usage: main forwarders collectors create [OPTIONS]
Create a collector.
Options:
--url TEXT Base URL to be used for API calls.
--region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
Select region.
--verbose Prints verbose output to the console.
-c, --credential_file TEXT Path of Service Account JSON. Default: /usr/
local/google/home/<user>/.chronicle_cl
i/chronicle_credentials.json
-h, --help Show this message and exit.
Sample output
$ chronicle_cli forwarders collectors create
================================================================================
Press Enter if you want to use the default value mentioned besides field description in [] brackets.
================================================================================
Enter Forwarder ID: a7e59660-959b-44e7-aa7e-baec820d01f4
(*) Collector Display Name: collector_4
========================================
======== Collector Configuration ======
========================================
(*) Collector Log Type (Type of logs collected): WINDOWS_DNS
Do you want to proceed with Collector Metadata? [y/N]: y
========================================
========== Collector Metadata ==========
========================================
Asset Namespace: test_namespace
========================================
=========== Forwarder Labels ===========
========================================
Labels (The ingestion metadata labels in 'key:value' format to apply to all logs ingested through this forwarder, as well as the resulting normalized data.)
Enter/Paste your content. On a new line, press Ctrl-D (Linux) / [Ctrl-Z + Enter (Windows)] to save it:
key1:value1
key2:value2
Do you want to proceed with Collector Regex Filters? [y/N]: y
========================================
======= Collector Regex Filters =======
========================================
Filter Description (Describes what is being filtered and why): desc1
Filter Regexp (The regular expression used to match against each incoming line): .*
Filter Behavior (Filter behavior to apply when a match is found)
Choose:
1. allow
2. block
: 1
You have selected allow
Do you want to add more Collector Regex Filters? [y/N]: n
Do you want to proceed with Collector Disk Buffer? [y/N]: n
Maximum Seconds per Batch (Maximum number of seconds between forwarder batch uploads) [10]:
Maximum Bytes per Batch (Maximum number of bytes queued before forwarder batch upload) [1048576]:
========================================
===== Configure Ingestion Settings =====
========================================
Choose:
1. File Settings
2. Kafka Settings
3. Pcap Settings
4. Splunk Settings
5. Syslog Settings
File Path (Path of file to monitor): path/to/file.txt
Preview changes:
- Press Up/b or Down/z keys to paginate.
- To switch case-sensitivity, press '-i' and press enter. By default, search
is case-sensitive.
- To search for specific field, press '/' key, enter text and press enter.
- Press 'q' to quit and confirm preview changes.
- Press `h` for all the available options to navigate the list.
=============================================================================
Config:
File settings:
File path: path/to/file.txt
Log type: WINDOWS_DNS
Max bytes per batch: 1048576
Max seconds per batch: 10
Metadata:
Asset namespace: test_namespace
Labels:
- key: key1
value: value1
- key: key2
value: value2
Regex filters:
Behavior: ALLOW
Description: desc1
Regexp: .*
Display name: collector_4
Do you want to create collector with this configuration? [y/N]: y
Creating collector...
Collector created successfully with Collector ID: 3a74b289-ccb4-4cee-9713-611a3362f48f
If the collector create fails, and you enter the same collector ID, then a prompt is displayed to retry the failed collector or restart the process. If you enter the collector ID which does not match with the failed collector ID, then the option to retry is not displayed and the normal process of updating the collector continues.
Sample output
$ chronicle_cli forwarders collectors create
Enter Forwarder ID: a7e59660-959b-44e7-aa7e-baec820d01f4
Looks like there was a failed create/update attempt for test_display.
Would you like to retry?
(*) Collector Display Name [test_display]:
========================================
======== Collector Configuration ======
========================================
(*) Collector Log Type (Type of logs collected) [WINDOWS_DNS]:
Do you want to proceed with Collector Metadata? [y/N]: y
========================================
========== Collector Metadata ==========
========================================
Asset Namespace [test]:
========================================
=========== Forwarder Labels ===========
========================================
Labels (The ingestion metadata labels in 'key:value' format to apply to all logs ingested through this forwarder, as well as the resulting normalized data.)
Enter/Paste your content. On a new line, press Ctrl-D (Linux) / [Ctrl-Z + Enter (Windows)] to save it:
[[{'key': 'k1', 'value': 'v1'}]]
Do you want to proceed with Collector Regex Filters? [y/N]: y
========================================
======= Collector Regex Filters =======
========================================
Filter Description (Describes what is being filtered and why)[old_desc]: desc2
Filter Regexp (The regular expression used to match against each incoming line) [.*]:
Filter Behavior (Filter behavior to apply when a match is found)
Choose:
1. allow
2. block
You have selected allow
Do you want to add more Collector Regex Filters? [y/N]: n
Do you want to proceed with Collector Disk Buffer? [y/N]: n
Maximum Seconds per Batch (Maximum number of seconds between forwarder batch uploads) [10]:
Maximum Bytes per Batch (Maximum number of bytes queued before forwarder batch upload) [1048576]:
========================================
===== Configure Ingestion Settings =====
========================================
Choose:
1. File Settings
2. Kafka Settings
3. Pcap Settings
4. Splunk Settings
5. Syslog Settings
: 1
File Path (Path of file to monitor) [path/to/file.txt]: path/to/file.txt
Preview changes:
- Press Up/b or Down/z keys to paginate.
- To switch case-sensitivity, press '-i' and press enter. By default, search
is case-sensitive.
- To search for specific field, press '/' key, enter text and press enter.
- Press 'q' to quit and confirm preview changes.
- Press `h` for all the available options to navigate the list.
=============================================================================
Config:
File settings:
File path: path/to/file.txt
Log type: WINDOWS_DNS
Max bytes per batch: 1048576
Max seconds per batch: 10
Metadata:
Asset namespace: test
Labels:
- key: k1
value: v1
Regex filters:
Behavior: ALLOW
Description: disc2
Regexp: .*
Display name: test_display
Do you want to create collector with this configuration? [y/N]: y
Creating collector...
Collector created successfully with Collector ID: b50a6b41-5476-41ee-ba7c-ce529ecffa62
get argument
To get details of an existing collectors, use the following command:
$ chronicle_cli forwarders collectors get --help
Usage: main forwarders collectors get [OPTIONS]
Get a collector using collector ID.
Options:
--url TEXT Base URL to be used for API calls.
--region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
Select region.
--verbose Prints verbose output to the console.
-c, --credential_file TEXT Path of Service Account JSON. Default: /usr/
local/google/home/<user>/.chronicle_cl
i/chronicle_credentials.json
-h, --help Show this message and exit.
Sample output
You must enter the collector ID interactively to get the collector details.
$ chronicle_cli forwarders collectors get
Enter Forwarder ID: a7e59660-959b-44e7-aa7e-baec820d01f4
Enter Collector ID: 3a74b289-ccb4-4cee-9713-611a3362f48f
Fetching collector details...
Collector Details:
ID: 3a74b289-ccb4-4cee-9713-611a3362f48f
Display name: collector_4
State: ACTIVE
Config:
Log type: WINDOWS_DNS
Metadata:
Asset namespace: test_namespace
Labels:
- key: key1
value: value1
- key: key2
value: value2
Regex filters:
- description: desc1
regexp: .*
behavior: ALLOW
- description: desc2
regexp: .*
behavior: BLOCK
Max seconds per batch: 10
Max bytes per batch: '1048576'
File settings:
File path: path/to/file.txt
list argument
To display list of all collectors, use following command:
chronicle_cli forwarders collectors list --help
Usage: main forwarders collectors list [OPTIONS]
List all collectors.
Options:
--url TEXT Base URL to be used for API calls.
--region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
Select region.
--verbose Prints verbose output to the console.
--export TEXT Export output to specified file path.
--file-format [TXT|CSV|JSON] Format of the file to be exported.
-c, --credential_file TEXT Path of Service Account JSON. Default: /usr/
local/google/home/<user>/.chronicle_cl
i/chronicle_credentials.json
-h, --help Show this message and exit.
Sample output
$ chronicle_cli forwarders collectors list
Enter Forwarder ID: a7e59660-959b-44e7-aa7e-baec820d01f4
Collector Details:
ID: 153e4077-cd49-4ce5-87aa-254d239b9dda
Display name: collector_2
State: ACTIVE
Config:
Log type: WINDOWS_DNS
Metadata:
Asset namespace: test
Labels:
- key: key1
value: value1
- key: key2
value: value2
Regex filters:
- description: desc1
regexp: .*
behavior: ALLOW
Disk buffer:
State: ACTIVE
Directory path: path/to/dir
Max file buffer bytes: '209'
Max seconds per batch: 10
Max bytes per batch: '1048576'
File settings:
File path: path/to/file.txt
================================================================================
Collector Details:
ID: b50a6b41-5476-41ee-ba7c-ce529ecffa62
Display name: test_display
State: ACTIVE
Config:
Log type: WINDOWS_DNS
Metadata:
Asset namespace: test
Labels:
- key: k1
value: v1
Regex filters:
- description: disc2
regexp: .*
behavior: ALLOW
- description: test
regexp: test
behavior: BLOCK
Disk buffer:
State: ACTIVE
Directory path: test
Max file buffer bytes: '55'
Max seconds per batch: 5
Max bytes per batch: '556676'
Syslog settings:
Protocol: TCP
Address: 1.2.3.4
Port: 3456
Buffer size: '65536'
Connection timeout: 60
Tls settings:
Certificate: test
Certificate key: test
Minimum tls version: '56'
Insecure skip verify: true
================================================================================
update argument
To update an existing collector, use the following command:
$ chronicle_cli forwarders collectors update --help
Usage: main forwarders collectors update [OPTIONS]
Update a collector using collector ID.
Options:
--url TEXT Base URL to be used for API calls.
--region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
Select region.
--verbose Prints verbose output to the console.
-c, --credential_file TEXT Path of Service Account JSON. Default: /usr/
local/google/home/<user>/.chronicle_cl
i/chronicle_credentials.json
-h, --help Show this message and exit.
Sample output
$ chronicle_cli forwarders collectors update
Enter Forwarder ID: a7e59660-959b-44e7-aa7e-baec820d01f4
Enter Collector ID: 3a74b289-ccb4-4cee-9713-611a3362f48f
(*) Collector Display Name [collector_4]:
========================================
======== Collector Configuration ======
========================================
(*) Collector Log Type (Type of logs collected) [WINDOWS_DNS]:
Do you want to proceed with Collector Metadata? [y/N]: y
========================================
========== Collector Metadata ==========
========================================
Asset Namespace [test_namespace]:
========================================
=========== Forwarder Labels ===========
========================================
Labels (The ingestion metadata labels in 'key:value' format to apply to all logs ingested through this forwarder, as well as the resulting normalized data.)
Enter/Paste your content. On a new line, press Ctrl-D (Linux) / [Ctrl-Z + Enter (Windows)] to save it:
[{'key1':'value1'},{'key2':'value2'}]
Do you want to proceed with Collector Regex Filters? [y/N]: y
========================================
======= Collector Regex Filters =======
========================================
Filter Description (Describes what is being filtered and why)[old_desc]: desc1
Filter Regexp (The regular expression used to match against each incoming line)[.*]: .*
Filter Behavior (Filter behavior to apply when a match is found)
Choose:
1. allow
2. block
You have selected allow
Do you want to add more Collector Regex Filters? [y/N]: y
Filter Description (Describes what is being filtered and why): desc2
Filter Regexp (The regular expression used to match against each incoming line): .*
Filter Behavior (Filter behavior to apply when a match is found)
Choose:
1. allow
2. block
: 2
You have selected block
Do you want to add more Collector Regex Filters? [y/N]: n
Do you want to proceed with Collector Disk Buffer? [y/N]: n
Maximum Seconds per Batch (Maximum number of seconds between forwarder batch uploads) [10]:
Maximum Bytes per Batch (Maximum number of bytes queued before forwarder batch upload) [1048576]:
========================================
===== Configure Ingestion Settings =====
========================================
Choose:
1. File Settings
2. Kafka Settings
3. Pcap Settings
4. Splunk Settings
5. Syslog Settings
File Path (Path of file to monitor) [path/to/file.txt]: path/to/file.txt
Do you want to update collector with this configuration? [y/N]: y
Updating collector...
Collector updated successfully with Collector ID: 3a74b289-ccb4-4cee-9713-611a3362f48f
If the collector update fails and you enter the same collector ID, then a prompt is displayed to retry the failed collector or to restart the process. If you enter a collector ID which does not match the failed collector ID, then the option to retry is not displayed and the normal process of updating the collector continues. The retry mechanism allows you to change option values provided in the previous failed attempt in an interactive way. Press Enter to reuse the same value for an option in the collector update flow.
...
Updating collector...
Do you want to update collector with this configuration? [y/N]: y
Error occurred while updating collector.
Response Code: 400.
Error: generic::invalid_argument: update collector (id: 3a74b289-ccb4-4cee-9713-611a3362f48f) for forwarder (id: a7e59660-959b-44e7-aa7e-baec820d01f4) for customer (id: ed19f037-2354-43df-bfbf-350362b45844): validation errors in request: generic::invalid_argument: filter's description is not specified: invalid argument
$ chronicle_cli forwarders collectors update
Enter Forwarder ID: a7e59660-959b-44e7-aa7e-baec820d01f4
Enter Collector ID: 3a74b289-ccb4-4cee-9713-611a3362f48f
Looks like there was a failed create/update attempt for collector_4.
Would you like to retry?
(*) Collector Display Name [collector_4]:
========================================
======== Collector Configuration ======
========================================
(*) Collector Log Type (Type of logs collected) [WINDOWS_DNS]:
Do you want to proceed with Collector Metadata? [y/N]: n
Do you want to proceed with Collector Regex Filters? [y/N]: y
========================================
======= Collector Regex Filters =======
========================================
Filter Description (Describes what is being filtered and why)[old_desc]: desc1
Filter Regexp (The regular expression used to match against each incoming line) [.*]:
Filter Behavior (Filter behavior to apply when a match is found)
Choose:
1. allow
2. block
You have selected allow
Do you want to add more Collector Regex Filters? [y/N]: n
Do you want to proceed with Collector Disk Buffer? [y/N]: n
Maximum Seconds per Batch (Maximum number of seconds between forwarder batch uploads) [10]:
Maximum Bytes per Batch (Maximum number of bytes queued before forwarder batch upload) [1048576]:
========================================
===== Configure Ingestion Settings =====
========================================
Choose:
1. File Settings
2. Kafka Settings
3. Pcap Settings
4. Splunk Settings
5. Syslog Settings
[1]:
File Path (Path of file to monitor) [path/to/file.txt]:
Do you want to update collector with this configuration? [y/N]: y
Updating collector...
Collector updated successfully with Collector ID: 3a74b289-ccb4-4cee-9713-611a3362f48f
delete argument
Use this argument to delete a collector using a collector ID. On execution, it asks for the ID of the collector to be deleted.
To delete an existing collector, use the following command:
$ chronicle_cli forwarders collectors delete --help
Usage: main forwarders collectors delete [OPTIONS]
Delete a collector using collector ID.
Options:
--url TEXT Base URL to be used for API calls.
--region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
Select region.
--verbose Prints verbose output to the console.
-c, --credential_file TEXT Path of Service Account JSON. Default: /usr/
local/google/home/<user>/.chronicle_cl
i/chronicle_credentials.json
-h, --help Show this message and exit.
Sample output
$ chronicle_cli forwarders collectors delete
Enter Forwarder ID: a7e59660-959b-44e7-aa7e-baec820d01f4
Enter Collector ID: 3a74b289-ccb4-4cee-9713-611a3362f48f
Collector (ID: 3a74b289-ccb4-4cee-9713-611a3362f48f) deleted successfully.
Options
Help (-h / --help)
Use -h or --help option to view usage/description for any command/option.
Sample Usage
$ chronicle_cli forwarders list -h
Usage: main forwarders list [OPTIONS]
List all forwarders
Options:
--url TEXT Base URL to be used for API calls.
--region [ASIA-NORTHEAST1|ASIA-SOUTH1|ASIA-SOUTHEAST1|AUSTRALIA-SOUTHEAST1|EUROPE|EUROPE-WEST2|EUROPE-WEST3|EUROPE-WEST6|ME-CENTRAL2|ME-WEST1|NORTHAMERICA-NORTHEAST2|US]
Select region.
--verbose Prints verbose output to the console.
--export TEXT Export output to specified file path.
--file-format [TXT|CSV|JSON] Format of the file to be exported.
-c, --credential_file TEXT Path of Service Account JSON. Default: /usr/
local/google/home/<user>/.chronicle_cl
i/chronicle_credentials.json
-h, --help Show this message and exit.
Credentials Path (-c or --credential-path)
This option allows to specify the path for the service account credentials to be used for the authentication of the user. If this option is not specified, then the Google SecOps CLI looks for credentials in the default path, i.e. ~/.chronicle_cli (inside a hidden directory named .chronicle_cli under the home directory).
Sample Usage
$ chronicle_cli forwarders list --credential-path=C:\chronicle_credentials.json
Verbose (--verbose)
This option allows you to print the details of the HTTP Request made and Response received.
Sample Usage
$ chronicle_cli forwarders list --verbose
Export (--export)
This option allows you to specify the file path where the output of the list
command will be exported. Both relative and absolute paths are supported.
Sample Usage
$ chronicle_cli forwarders list --export=$HOME/listForwarderssResponse
File Format (--file-format)
This option allows you to specify the file format of the contents exported with
the list command. There are three formats supported: CSV, JSON and TXT. If this
option is not specified with the --export option, then CSV format would be used as default.
Sample Usage
$ chronicle_cli forwarders list --export=$HOME/listForwardersResponse --file-format=JSON
Sample output
JSON format
{
"forwarders": [
{
"name": "55a77e24-9d16-4638-8940-0ef8071ed849",
"displayName": "new",
"config": {
"uploadCompression": true,
"metadata": {
"assetNamespace": "test",
"labels": [
{
"key": "k",
"value": "v"
},
{
"key": "k1",
"value": "v1"
}
]
},
"regexFilters": [
{
"description": "desc1",
"regexp": ".*",
"behavior": "ALLOW"
}
],
"serverSettings": {
"gracefulTimeout": 15,
"drainTimeout": 10,
"httpSettings": {
"port": 8080,
"host": "0.0.0.0",
"readTimeout": 3,
"readHeaderTimeout": 3,
"writeTimeout": 3,
"idleTimeout": 3,
"routeSettings": {
"availableStatusCode": 204,
"readyStatusCode": 204,
"unreadyStatusCode": 503
}
},
"state": "ACTIVE"
}
},
"state": "ACTIVE",
"collectors": {
"Collector [3e8243c3-7ff2-4ede-89fe-16410ffe03bd]": {
"name": "3e8243c3-7ff2-4ede-89fe-16410ffe03bd",
"displayName": "cre_test_2",
"state": "ACTIVE",
"config": {
"logType": "WINDOWS_DNS",
"metadata": {
"assetNamespace": "test",
"labels": [
{
"key": "k",
"value": "v"
}
]
},
"regexFilters": [
{
"description": "desc1",
"regexp": ".*",
"behavior": "ALLOW"
}
],
"diskBuffer": {
"state": "ACTIVE",
"directoryPath": "23",
"maxFileBufferBytes": "33"
},
"maxSecondsPerBatch": 10,
"maxBytesPerBatch": "1048576",
"fileSettings": {
"filePath": "path/file.txt"
}
}
}
}
}
]
}
CSV format
1. {file_name}_forwarders.csv
2. {file_name}_collectors.csv
Sample content of file:
{file_name}_forwarders.csv:
Name,Display name,Forwarder state,[CONFIG] Upload compression,[CONFIG][METADATA] Asset namespace,[CONFIG][METADATA] Labels,[CONFIG] Regex filters,[CONFIG][SERVER_SETTINGS] Server state,[CONFIG][SERVER_SETTINGS] Graceful timeout,[CONFIG][SERVER_SETTINGS] Drain timeout,[CONFIG][SERVER_SETTINGS][HTTP_SETTINGS] Port,[CONFIG][SERVER_SETTINGS][HTTP_SETTINGS] Host,[CONFIG][SERVER_SETTINGS][HTTP_SETTINGS] Read timeout,[CONFIG][SERVER_SETTINGS][HTTP_SETTINGS] Read header timeout,[CONFIG][SERVER_SETTINGS][HTTP_SETTINGS] Write timeout,[CONFIG][SERVER_SETTINGS][HTTP_SETTINGS] Idle timeout,[CONFIG][SERVER_SETTINGS][HTTP_SETTINGS][ROUTE_SETTINGS] Available status code,[CONFIG][SERVER_SETTINGS][HTTP_SETTINGS][ROUTE_SETTINGS] Ready status code,[CONFIG][SERVER_SETTINGS][HTTP_SETTINGS][ROUTE_SETTINGS] Unready status code
0593ba21-a1c7-4279-b429-bc8df959bd59,test,ACTIVE,True,test,"k1: v1
k2: v2
",,,,,,,,,,,0,0,0
094c9e41-e7c8-407a-8b9a-eb34d608a609,test,ACTIVE,True,te,"k1: v1
k2: v2
",,,,,,,,,,,0,0,0
1189f869-5f3c-4ec7-ba48-9c80e33aadf0,test,ACTIVE,True,test,"key1: value1
key2: value2
",,ACTIVE,15,10,8080,0.0.0.0,3,4,5,8,204,204,500
{file_name}_collectors.csv
Forwarder ID,Name,Display Name,Collector state,[CONFIG] Log type,[CONFIG] Max seconds per batch,[CONFIG] Max bytes per batch,[CONFIG][METADATA] Asset namespace,[CONFIG][METADATA] Labels,[CONFIG] Regex filters,[CONFIG][DISK_BUFFER] State,[CONFIG][DISK_BUFFER] Directory path,[CONFIG][DISK_BUFFER] Max file buffer bytes,[CONFIG][FILE_SETTINGS] File path,[CONFIG][KAFKA_SETTINGS][AUTHENTICATION] username,[CONFIG][KAFKA_SETTINGS][AUTHENTICATION] password,[CONFIG][KAFKA_SETTINGS] Topic,[CONFIG][KAFKA_SETTINGS] Group id,[CONFIG][KAFKA_SETTINGS] Timeout,[CONFIG][KAFKA_SETTINGS] Brokers,[CONFIG][KAFKA_SETTINGS][TLS_SETTINGS] Certificate,[CONFIG][KAFKA_SETTINGS][TLS_SETTINGS] Certificate key,[CONFIG][KAFKA_SETTINGS][TLS_SETTINGS] Minimum tls version,[CONFIG][KAFKA_SETTINGS][TLS_SETTINGS] Insecure skip verify,[CONFIG][PCAP_SETTINGS] Network interface,[CONFIG][PCAP_SETTINGS] Bpf,[CONFIG][SPLUNK_SETTINGS][AUTHENTICATION] username,[CONFIG][SPLUNK_SETTINGS][AUTHENTICATION] Password,[CONFIG][SPLUNK_SETTINGS] Host,[CONFIG][SPLUNK_SETTINGS] Port,[CONFIG][SPLUNK_SETTINGS] Minimum window size,[CONFIG][SPLUNK_SETTINGS] Maximum windows size,[CONFIG][SPLUNK_SETTINGS] Query string,[CONFIG][SPLUNK_SETTINGS] Query mode,[CONFIG][SPLUNK_SETTINGS] Cert ignored,[CONFIG][SYSLOG_SETTINGS] Protocol,[CONFIG][SYSLOG_SETTINGS] Address,[CONFIG][SYSLOG_SETTINGS] Port,[CONFIG][SYSLOG_SETTINGS] Buffer size,[CONFIG][SYSLOG_SETTINGS] Connection timeout,[CONFIG][SYSLOG_SETTINGS][TLS_SETTINGS] Certificate,[CONFIG][SYSLOG_SETTINGS][TLS_SETTINGS] Certificate key,[CONFIG][SYSLOG_SETTINGS][TLS_SETTINGS] Minimum tls version,[CONFIG][SYSLOG_SETTINGS][TLS_SETTINGS] Insecure skip verify
1189f869-5f3c-4ec7-ba48-9c80e33aadf0,03d28371-1bcb-4b28-9364-18412de1f827,collector_2,ACTIVE,WINDOWS_DNS,10,1048576,collector_update,"key1: value1
key2: value2
",,ACTIVE,path/file.txt,23,path/to/file.txt,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
1189f869-5f3c-4ec7-ba48-9c80e33aadf0,8ba8278c-1eef-4a72-a45a-491463768c70,col_3,ACTIVE,WINDOWS_DNS,10,1048576,test,"k1: v1
",,ACTIVE,path/to/file,233,path,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
55a77e24-9d16-4638-8940-0ef8071ed849,3e8243c3-7ff2-4ede-89fe-16410ffe03bd,cre_test_2,ACTIVE,WINDOWS_DNS,10,1048576,test,"k: v
",,ACTIVE,23,33,path/file.txt,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
Region (--region)
You can select the region by passing the --region flag when you run a command.
For more information on how to set the default region, see Set default region.
Troubleshooting
Error Codes
This section shows the output displayed on the console against different types of response codes received from API response.
Refer to the following tables to view the output for each command:
Get command
| Response Code | Console Output |
| 404 | {Forwarder|Collector} does not exist.
|
| 400 | Invalid {Forwarder|Collector} ID. Please enter valid {Forwarder|Collector} ID.
|
| Any other response code | Error while fetching {Forwarder|Collector}.
Response Code: {status code}
Error: {error message}
|
List command
| Response Code | Console Output |
| Any response code other than 200 | Error:
Response Code: {status code}
Error: {error message}
|
Create command
| Response Code | Console Output |
| Any response code other than 200 | Error occurred while creating {forwarder|collector}
Response Code: {status code}
Error: {error message}
|
Update command
| Response Code | Console Output |
| Any response code other than 200 | Error occurred while updating {forwarder|collector}. Response Code: {status code}
Error: {error message}
|
Delete command
| Response Code | Console Output |
| 404 | {Forwarder|Collector} does not exist.
|
| 400 | Invalid {Forwarder|Collector} ID. Please enter valid {Forwarder|Collector} ID.
|
| Any other response code | Error occurred while deleting {Forwarder|Collector}.
Response Code: {status code}
Error: {error message}
|
Other errors or exceptions
| Exception | Console Output |
| KeyError | Failed to find key {key name} in the response.
|
| Exception | Failed with exception: {exception details}
|
| Credential file missing | Failed with exception: [Errno 2] No such file or directory: '/usr/local/google/home/ You need to place credentials in the expected directory. Refer to Installation. |
For any other queries or issues with the Google Security Operations CLI, please reach out to Google Security Operations Support.
BigQuery data access workflows
Google Security Operations supports self-service access to Google Security Operations data in BigQuery. You can use the Google Security Operations CLI to grant Identity and Access Management (IAM) roles that give the following permissions for a user email:
roles/bigquery.dataViewerroles/bigquery.jobUserroles/storage.objectViewer
The email must be a Google Accounts and ID Administration (GAIA) user email address of a Google SecOps customer.
For more information about these roles, see Exporting table data.
Commands
bigquery command
The bigquery command takes the provide_access argument.
Usage syntax:
$ chronicle_cli bigquery ARGUMENT [OPTIONS]
Arguments
provide_access argument
Prompts you to enter a user email address. The email must be a Google Accounts and ID Administration (GAIA) user email address of a Google SecOps Security customer. The user will receive the required IAM roles to do the following:
- Read data and metadata from BigQuery tables (
roles/bigquery.dataViewer) - Run queries on BigQuery table data (
roles/bigquery.jobUser) - Read data in Google Cloud Storage buckets(
roles/storage.objectViewer)
Sample usage
$ chronicle_cli bigquery provide_access
$ Enter email: xyz@gmail.com
Success response
Providing BigQuery access...
Access provided to email: xyz@gmail.com
Error response
Providing BigQuery access...
Error while providing access:
Response code: 400
Options
Help (-h / --help)
Use -h or --help option to view usage/description for any command/option.
Troubleshooting
This section shows the output displayed on the console against different types of response codes received from API response.
provide_access argument response codes
| Response Code | Console Output |
| 400 | Email does not exist. |
| Any other response code | Error while fetching feed.
Response Code: {status code}
Error: {error message}
|