UDM Search Best Practices

Supported in:

This document describes Google's recommended best practices for conducting searches using UDM Search. UDM searches can require substantial computational resources to complete if they are not constructed carefully. Performance also varies depending on the size and complexity of the data in your Google Security Operations instance.

Each condition must be in the form of udm-field operator value.

For example: principal.hostname = "win-server"

Always try to narrow your time range to the minimum necessary. Google Security Operations can ingest a tremendous amount of data, so limiting the breadth of that data while conducting a search can substantially improve search performance.

You can use regular expressions when conducting UDM searches:

  • Use AND, OR, and NOT.
  • AND is assumed in the absence of the other operators.
  • Use parentheses to modify the order of precedence. There is a maximum limit of 169 logical operators (OR, AND, and NOT) that can be used in the parentheses.
  • Depending on the field type, field operators can include: = != >= > < <=

Alternatively, you can also use the reference lists.

Using nocase as a search modifier

nocase can be used as a modifier to ignore capitalization.

For example, the following search is invalid:

target.user.userid = "TIM.SMITH" nocase

Regular expressions don't work for enumerated fields

You can't use regular expressions for enumerated fields (fields with a range of predefined values) like metadata.event_type or network.ip_protocol.

For example, the following search is invalid:

metadata.eventtype = /NETWORK*/

However, the following search is valid (and approximates what was attempted above):

(metadata.event_type = "NETWORK_CONNECTION" or metadata.event_type = "NETWORK_DHCP")

Using any and all operators in the Events field

In UDM search, some fields are labeled as repeated, which means they are lists of values or other types of messages. Unlike YARA-L, repeated fields in UDM search are always treated with the any operator by default, with no option to specify all in your search.

When the any operator is used, the predicate is evaluated as true if any value in the repeated field satisfies the condition. For example, if you search for principal.ip != "1.2.3.4" and events in your search include both principal.ip = "1.2.3.4" and principal.ip = "5.6.7.8", the search will generate matches. This expands your search to include results that match any of the operators instead of matching with all of them.

Each element in the repeated field is treated individually. If the repeated field is found in events in the search, the events are evaluated for each element in the field. This can cause unexpected behavior, especially when searching using the != operator.

When using the any operator, the predicate is evaluated as true if any value in the repeated field satisfies the condition.

Timestamps use Unix epoch time

Timestamp fields are matched using Unix epoch time (number of seconds that have passed since Thursday 1 January 1970 00:00:00).

When searching for a specific timestamp, the following (in epoch time) is valid:

metadata.ingested_timestamp.seconds = 1660784400

The following timestamp is invalid:

metadata.ingested_timestamp = "2022-08-18T01:00:00Z"

There are certain fields that are excluded from filters, including the following:

  • metadata.id
  • metadata.product_log_id
  • *.timestamp

Since these fields tend to have unique values, displaying them creates more "noise" than value in the UDM Search interface.