Collect F5 BIG-IP APM logs

Supported in:

Google SecOps SIEM

Note: This feature is covered by Pre-GA Offerings Terms of the Google Security Operations Service Specific Terms. Pre-GA features might have limited support, and changes to pre-GA features might not be compatible with other pre-GA versions. For more information, see the Google Security Operations Technical Support Service guidelines and the Google Security Operations Service Specific Terms.

This document describes how you can collect F5 BIG-IP Access Policy Manager (APM) logs by using a Google Security Operations forwarder.

For more information, see Data ingestion to Google Security Operations.

An ingestion label identifies the parser which normalizes raw log data to structured UDM format. The information in this document applies to the parser with the F5_BIGIP_APM ingestion label.

Configure F5 BIG-IP APM

1. Sign in to the BIG-IP configuration utility portal using administrator credentials.
2. Select Main > System > Logs > Configuration > Remote logging.

3. In the Properties section, do the following:

   • In the Remote IP field, enter the Google Security Operations forwarder IP address.
   • In the Remote port field, enter a high port number.

4. Click Add.

5. Click Update.

   For logs from APM, only the Berkeley Software Distribution (BSD) syslog format is supported.

   Based on the signatures in the APM, the collector processes only APM logs. The F5 BIG-IP APM event collector supports multi-threading logs from LTM 11.6 to 12.1.1 device also.

   If you are using iRule, use the recommended format of iRule. Google Security Operations supports the following iRule format only:

   # log_header_requests
   ###################################################################################
   #################################################
   # Purpose: logs header information to Local Traffic log
   # #
   #
   # Update-Log Date By Description
   # Created 02/07/2020 E01961 Initial implementation
   #
   #
   ###################################################################################
   ################################################
   when HTTP_REQUEST {
   set LogString "Client [IP::client_addr]:[TCP::client_port] -> [HTTP::host]
   [HTTP::uri]"
   log local5. "================="
   log local5. "$LogString (request)"
   foreach aHeader [HTTP::header names] {
   log local5. "$aHeader: [HTTP::header value $aHeader]"
   }
   # set UserID [URI::query "?[HTTP::payload]" "UserID"]
   # log local0. "User $UserID attempted login from [IP::client_addr] and referer:
   [HTTP::header "Referer"]"
   # log local0. "============================================="
   }
   when HTTP_RESPONSE {
   log local5. "=================="
   log local5. "$LogString (response) - status: [HTTP::status]"
   foreach aHeader [HTTP::header names] {
   log local5. "$aHeader: [HTTP::header value $aHeader]"
   }
   # log local0. "============================================="
   

Configure F5 BIG-IP DNS

To configure F5 BIG-IP DNS, do the following tasks:

• Create a pool of remote logging servers.
• Create a remote high-speed log destination.
• Create a formatted remote high-speed log destination.
• Create a publisher.
• Create a custom DNS logging profile.
• Add a DNS logging profile to the listener.

Create a pool of remote logging servers

1. On the Main tab, select DNS > Delivery > Load balancing > Pools or local traffic > Pools.
2. In the Pool list window that appears, click Create.
3. In the New pool window that appears, in the Name field, provide a unique name for the pool.
4. In the New members section, add the IP address for each remote logging server that you want to include in the pool:
   1. In the Address field, enter the Google Security Operations forwarder IP address or select a node address from the node list.
   2. In the Service port field, type a service number or select a service name from the list. Ensure that you have configured the correct remote logging port.
5. Click Add, and then click Finished.

Create a remote high-speed log destination

1. On the Main tab, select System > Logs > Configuration > Log destinations.
2. In the Log destinations window that appears, click Create.
3. In the Name field, provide a unique and identifiable name for this destination.
4. In the Type list, select Remote high-speed log.
5. In the Pool name list, select the pool of remote log servers to which you want the BIG-IP system to send log messages.
6. In the Protocol list, select the protocol used by the high-speed logging pool members.
7. Click Finished.

Create a formatted remote high-speed log destination

1. On the Main tab, select System > Logs > Configuration > Log Destinations.
2. In the Log destinations window that appears, click Create.
3. In the Name field, provide a unique and identifiable name for this destination.
4. In the Type list, select a formatted logging destination as Remote syslog. The BIG-IP system is now configured to send a formatted string of text to the log servers.
5. In the Type list, select a format for the logs.
6. On the Forward To tab, select High-speed log destination list and then select the destination that points to a pool of remote syslog servers to which you want the BIG-IP system to send log messages.
7. Click Finished.

Create a publisher

1. On the Main tab, select System > Logs > Configuration > Log publishers.
2. In the Log publishers window that appears, click Create.
3. In the Name field, provide a unique and identifiable name for the publisher.
4. In the Log publisher list, from the available list select the destination created previously.
5. To move the destination to the selected list, click << Move.
6. If you are using a formatted destination, select the newly-created destination that matches your log servers, such as Remote syslog, Splunk, or ArcSight.
7. Click Finished.

Create a custom DNS logging profile

1. On the Main tab, select DNS > Delivery > Profiles > Other DNS Logging or Local traffic > Profiles > Others > DNS logging.
2. In the DNS Logging profile list window that appears, click Create.
3. In the Name field, provide a unique name for the profile.
4. In the Log publisher list, select a destination to which the BIG-IP system sends DNS log entries.
5. If you want the BIG-IP system:
   • To log all DNS queries, from the Log queries setting, ensure that the enabled checkbox is selected.
   • To log all DNS responses, from the Log responses setting, select the enabled checkbox.
   • To include the query ID sent by the client in the log messages, from the Include query ID setting, select the enabled checkbox.
6. Click Finished.

Add a DNS logging profile to the listener

1. On the Main tab, select DNS > Delivery > Listeners > DNS listener.
2. In the Service section, from the DNS profile list, select the DNS profile that you previously configured.
3. Click Update.

Configure the Google Security Operations forwarder to ingest F5 BIG-IP APM logs

1. In the Google Security Operations menu, select Settings > Forwarders > Add new forwarder.
2. In the Forwarder name field, enter a unique name for the forwarder.
3. Click Submit. The forwarder is added and the Add collector configuration window appears.
4. In the Collector name field, enter a unique name for the collector.
5. In the Log type field, specify F5 BIGIP Access Policy Manager.
6. Select Syslog as the Collector type.
7. Configure the following input parameters:
   • Protocol: specify the protocol.
   • Address: specify the target IP address or hostname where the collector resides and addresses to the syslog data.
   • Port: specify the target port where the collector resides and listens for syslog data.
8. Click Submit.

For more information about the Google Security Operations forwarders, see Manage forwarder configurations through the Google Security Operations UI.

If you encounter issues when you create forwarders, contact Google Security Operations support.

Field mapping reference

This F5 BIG-IP APM parser extracts fields from syslog messages, categorizing them based on the application source (tmsh, tmm, apmd, httpd, or other). It then maps these extracted fields to the UDM, handling various log formats and enriching the data with metadata like severity, location, and user information.

UDM Mapping Table

Log Field	UDM Mapping	Logic
application	principal.application	The value is taken from the application field extracted by the grok filter.
bytes_in	network.received_bytes	The value is taken from the bytes_in field extracted by the grok filter and converted to unsigned integer.
bytes_out	network.sent_bytes	The value is taken from the bytes_out field extracted by the grok filter and converted to unsigned integer.
cmd_data	principal.process.command_line	The value is taken from the cmd_data field extracted by the kv filter.
destination_ip	target.ip	The value is taken from the destination_ip field extracted by the grok filter.
destination_port	target.port	The value is taken from the destination_port field extracted by the grok filter and converted to integer.
folder	principal.process.file.full_path	The value is taken from the folder field extracted by the kv filter.
geoCountry	principal.location.country_or_region	The value is taken from the geoCountry field extracted by the grok filter.
geoState	principal.location.state	The value is taken from the geoState field extracted by the grok filter.
inner_msg	security_result.description	The value is taken from the inner_msg field extracted by the grok filter when no other specific description is available.
ip_protocol	network.ip_protocol	The value is taken from the ip_protocol field extracted by the grok filter.
principal_hostname	principal.hostname	The value is taken from the principal_hostname field extracted by the grok filter.
principal_ip	principal.ip	The value is taken from the principal_ip field extracted by the grok filter.
process_id	principal.process.pid	The value is taken from the process_id field extracted by the grok filter.
role	user_role.name	The value is taken from the role field extracted by the grok filter. If the role field contains "admin" (case-insensitive), the value is set to "ADMINISTRATOR".
severity	security_result.severity_details	The original value from the syslog message is stored here. The value is derived from the severity field using conditional logic: CRITICAL -> CRITICAL ERR -> ERROR ALERT, EMERGENCY -> HIGH INFO, NOTICE -> INFORMATIONAL DEBUG -> LOW WARN -> MEDIUM
source_ip	principal.ip	The value is taken from the source_ip field extracted by the grok filter.
source_port	principal.port	The value is taken from the source_port field extracted by the grok filter and converted to integer.
status	security_result.summary	The value is taken from the status field extracted by the kv filter.
timestamp	metadata.event_timestamp, timestamp	The value is taken from the timestamp field extracted by the grok filter and parsed into a timestamp object. The timestamp field in the top level event object also gets this value.
user	principal.user.userid	The value is taken from the user field extracted by the grok filter, after removing "id\" or "ID\" prefixes. The value is derived based on the presence of other fields: If user exists: USER_UNCATEGORIZED If source_ip and destination_ip exist: NETWORK_CONNECTION If principal_ip or principal_hostname exist: STATUS_UPDATE Otherwise: GENERIC_EVENT Hardcoded to "BIGIP_APM". Hardcoded to "F5". If the result field is "failed", the value is set to "BLOCK".

Changes

2023-06-06

• Newly created parser.