Manage prebuilt and custom parsers

Supported in:

Google SecOps SIEM

This document describes how to: This document describes how to:

• Create and manage custom parsers.
• Get early access to upcoming prebuilt parser updates initiated by Google Security Operations.
• Extend mapping instructions by creating a parser extension for a prebuilt or custom parser.
• Control access to parser management.
• Create and manage custom parsers.
• Get early access to upcoming prebuilt parser updates initiated by Google Security Operations.
• Extend mapping instructions by creating a parser extension for a prebuilt or custom parser.
• Control access to parser management.

Types of parsers: Types of parsers:

original log data to [UDM](/chronicle/docs/event-processing/udm-overview) fields.

Parser type	Description
Prebuilt	Parsers that are created by Google Security Operations and contain built-in data mapping instructions for transforming original log data to [UDM](/chronicle/docs/event-processing/udm-overview) fields.
Prebuilt extended	A prebuilt parser created by customers with additional mapping instructions to extract additional data from an original raw log and insert it into the UDM record.
Custom	Parsers that are created by customers with custom data mapping instructions for transforming original log data to UDM fields.
Custom extended	A custom parser created by customers with additional mapping instructions using a parser extension to extract additional data from an original raw log and insert it into the UDM record.

Manage prebuilt parser updates

Manage prebuilt parser updates

Google Security Operations usually updates its prebuilt parsers during the fourth week of each month. These updates are first made available to customers for early access and testing. As upcoming parser updates become available, they are marked as Pending update in the parser list. You can examine the difference between the earlier and the newer parser versions, or make the parser update active early to test it, or skip the update and create a custom parser. Google Security Operations usually updates its prebuilt parsers during the fourth week of each month. These updates are first made available to customers for early access and testing. As upcoming parser updates become available, they are marked as Pending update in the parser list. You can examine the difference between the earlier and the newer parser versions, or make the parser update active early to test it, or skip the update and create a custom parser.

To view the pending, update do the following:

1. Log in to your Google Security Operations instance.

2. From the application menu, select Settings > Parsers.

3. Click Filter.

4. Select Prebuilt, Active, and Prebuilt Extended from the list.

   A list of active (default), prebuilt parsers displays. Upcoming parser updates are marked as Pending in the Update column.

5. Click Menu and select View pending update from the list.

   The Compare parsers page appears. Here, you can view the following:

   • The code difference between current and the upcoming parser version.

   • The changelogs in the Change logs tab.

   • The generated UDM event for the sampled raw log.

   • The date and time the parser was created.

   • The date and time the parser code was last updated.

   You can either make the parser update active early, skip the update and create a custom parser, or wait for the update to be auto-applied during the fourth week of the month.

Make the parser update active early

The parser management feature lets you make the parser update active early. For example, if you want to test it.

To make the parser update active early, follow these steps:

1. On the Compare parsers page, click Make parser update active.

   The Confirm parser update dialog appears.

2. Click Confirm.

   The parser is activated for the normalization process after 20 minutes.

Note: This parser version remains active until the next release cycle.

Skip prebuilt parser updates

To skip the current and future prebuilt parser updates, create a custom parser as follows:

1. On the Compare parsers page, click Skip update.

   The Skip update and create custom parser window appears.

2. Click Create custom parser.

3. For the Type of parser to start with, select either the current Prebuilt Parser, or the Pending Parser Update.

4. Click Create.

   Note: If a custom parser already exists for this log source, you cannot create another one. The following message is displayed, A custom parser already exists for this log source.

   The selected version is activated for the normalization process after 20 minutes. It appears as Custom and Active in the parsers list on the Parsers page. The earlier prebuilt version appears as Prebuilt and Inactive.

   Note: All future prebuilt parser updates remain visible, but they're not applied unless you disable the custom parser and revert to the prebuilt version.

Revert an early update of the prebuilt parser

If you activated the parser update early, you can still revert to the previous version until the fourth week of the month, when the update is automatically activated.

To switch back to the previous parser version, follow these steps:

1. From the application menu, select Settings > Parsers.

2. Click Menu against the parser that you want to revert.

3. Click View.

   The View prebuilt parser page appears.

4. Click Revert to previous version.

   The Revert to previous dialog appears. You can click Compare Parsers on the dialog to see the difference between the current and the previous versions.

5. Click Confirm to revert the parser to its previous version.

   The parser is reverted to its previous version after 20 minutes.

Custom parsers

Google Security Operations provides the flexibility to create a custom parser for a number of reasons, including:

• Create a custom parser for a log type that does not have a prebuilt parser. Create an entirely new parser directly from the raw log, or use an existing parser as a base for the new custom parser.
• You can create a custom parser to skip prebuilt parser updates.

Custom parsers are displayed in the parsers list.

Create a custom parser based on mapping instructions

You can create a custom parser by writing code that converts the original raw log to a UDM record. For the structure of a parser, see Overview of log parsing and Parser syntax reference for information about the syntax. When creating a parser, make sure the data mapping instructions populate as many important UDM fields as possible.

1. Go to SIEM Settings.

2. Go to SIEM Settings.

3. Click Create Parser.

4. Select an appropriate log source from the Log Source list.

5. Select Start with Raw Logs Only to create a new parser according to your requirements.

6. Click Create.

7. Enter code in the Parser Code Terminal. For more information, see Create a code snippet mapping instruction.

8. Enter code in the Parser Code Terminal. For more information, see Create a code snippet mapping instruction.

9. Optional: Click to edit the existing raw log or copy.

10. Optional: Click to load the latest raw log.

11. Click Preview to view the UDM output. An error message is displayed if the code is incorrect.

    In the preview, you can use the statedump filter plugin to validate the internal state of a parser. For more information, see Validate data using statedump plugin.

12. Click Validate to validate the custom parser.

    The validation process may take a few minutes, so we recommend that you preview the custom parser first, make changes if required, and then validate the custom parser.

13. Click Submit.

    The parser is activated for the normalization process after 20 minutes. The parser is activated for the normalization process after 20 minutes.

Create a custom parser from an existing parser

Create a custom parser from an existing parser

You can use an existing parser as a template to create a new parser. You can create a custom parser using the code approach only. To create a custom parser from an existing parser, follow these steps:

1. From the application menu, select Settings > Parsers.

2. Click Create Parser.

3. Select an appropriate log source from the Log Source list.

4. Select Start with an Existing Prebuilt Parser to use an existing parser as a base to create a new custom parser.

   Note: If the selected log source does not have a prebuilt parser then this option is not displayed.

5. Click Create.

6. Edit the code in the Parser Code Terminal. For more information, see Create a code snippet mapping instruction.

7. Optional: Click to edit the raw log.

8. Optional: Click to refresh the raw log.

9. As you add code to build the parser, click Preview to view the UDM output. An error message is displayed if the code is incorrect.

   In the preview, you can use the statedump filter plugin to validate the internal state of a parser. For more information, see Validate data using statedump plugin.

10. Click Validate to validate the custom parser.

    The validation process may take a few minutes, so we recommend that you preview the custom parser first, make changes if required, and then validate the custom parser.

11. Click Submit.

    The parser is activated for the normalization process after 20 minutes. The parser is activated for the normalization process after 20 minutes.

Make a custom parser inactive

1. From the application menu, select Settings > Parsers.

2. Click Menu against the parser that you want to make inactive and select Make inactive from the list.

   The Make parser inactive dialog appears.

3. Click Make inactive.

The custom parser is deactivated and the current prebuilt parser version is activated after 20 minutes. The prebuilt parser now becomes the default parser. The custom parser is deactivated and the current prebuilt parser version is activated after 20 minutes. The prebuilt parser now becomes the default parser.

Delete a custom parser

1. From the application menu, select Settings > Parsers.

2. Click Menu against the custom parser that you want to delete and select Delete from the list. Note: You cannot delete a prebuilt parser.

3. Click Menu against the custom parser that you want to delete and select Delete from the list. Note: You cannot delete a prebuilt parser.

   The Delete custom parser dialog appears.

4. Click Delete.

The custom parser is deleted and the current prebuilt parser version is activated after 20 minutes. The custom parser is deleted and the current prebuilt parser version is activated after 20 minutes.

Create an extension

Parser extensions provide a flexible way to extend the capabilities of existing prebuilt (default) parsers and custom parsers. They don't replace prebuilt or custom parsers; instead they enable the seamless extraction of additional fields from the original raw log into the UDM record. A parser extension is different from a custom parser. To create a parser extension, see Using parser extensions. Parser extensions provide a flexible way to extend the capabilities of existing prebuilt (default) parsers and custom parsers. They don't replace prebuilt or custom parsers; instead they enable the seamless extraction of additional fields from the original raw log into the UDM record. A parser extension is different from a custom parser. To create a parser extension, see Using parser extensions.

Control access to parser management

By default, users with Administrator and Editor roles can manage parser updates. New permissions can be granted to control who can view and manage these updates. For more information about managing users and groups, or assigning roles, see the role-based access control user guide.