Collect Azure Firewall logs

Supported in:

Google SecOps SIEM

Note: This feature is covered by Pre-GA Offerings Terms of the Google Security Operations Service Specific Terms. Pre-GA features might have limited support, and changes to pre-GA features might not be compatible with other pre-GA versions. For more information, see the Google Security Operations Technical Support Service guidelines and the Google Security Operations Service Specific Terms.

This document explains how to export Azure Firewall logs to Google Security Operations using Azure Storage Account. The parser first attempts to process the input as JSON, extracting data from the Records field. If the Record field is empty, the parser then uses a series of Grok patterns and conditional statements to extract relevant fields from the message, handling different formats and variations in the Azure Firewall logs.

Before you begin

• Ensure that you have A Google SecOps instance.
• Ensure that you have an active Azure tenant.
• Ensure that you have privileged access to Azure.

Configure Azure Storage Account

1. In the Azure console, search for Storage accounts.
2. Click + Create.
3. Specify values for the following input parameters:
   • Subscription: select the subscription.
   • Resource Group: select the resource group.
   • Region: select the region.
   • Performance: select the performance (Standard recommended).
   • Redundancy: select the redundancy (GRS or LRS recommended).
   • Storage account name: enter a name for the new storage account.
4. Click Review + create.
5. Review the overview of the account and click Create.
6. From the Storage Account Overview page, select the Access keys submenu in Security + networking.
7. Click Show next to key1 or key2
8. Click Copy to clipboard to copy the key.
9. Save the key in a secure location for later use.
10. From the Storage Account Overview page, select the Endpoints submenu in Settings.
11. Click Copy to clipboard to copy the Blob service endpoint URL (for example, https://<storageaccountname>.blob.core.windows.net)
12. Save the endpoint URL in a secure location for later use.

Configure Log Export for Azure Firewalls Logs

1. Sign in to the Azure Portal using you privileged account.
2. Go to Firewalls and select the required firewall.
3. Select Monitoring > Diagnostic Services.
4. Click + Add diagnostic setting.
   • Enter a descriptive name for the diagnostic setting.
5. Select allLogs.
6. Select the Archive to a storage account checkbox as the destination.
   • Specify the Subscription and Storage Account.
7. Click Save.

Configure a feed in Google SecOps to ingest the Azure Firewall logs

1. Go to SIEM Settings > Feeds.
2. Click Add new.
3. In the Feed name field, enter a name for the feed (for example, Azure Firewall Logs).
4. Select Microsoft Azure Blob Storage as the Source type.
5. Select Azure Firewall as the Log type.
6. Click Next.

7. Specify values for the following input parameters:

   • Azure URI: the blob endpoint URL.
     • ENDPOINT_URL/BLOB_NAME
       • Replace the following:
       • ENDPOINT_URL: the blob endpoint URL (https://<storageaccountname>.blob.core.windows.net)
       • BLOB_NAME: the name of the blob (such as, <logname>-logs)
   • URI is a: select the URI TYPE according to log stream configuration (Single file | Directory | Directory which includes subdirectories).

   • Source deletion options: select the deletion option according to your preference.

     Note: If you select the Delete transferred files or Delete transferred files and empty directories option, make sure that you granted appropriate permissions to the service account.

   • Shared key: the access key to the Azure Blob Storage.

   • Asset namespace: the asset namespace.

   • Ingestion labels: the label to be applied to the events from this feed.

8. Click Next.

9. Review your new feed configuration in the Finalize screen, and then click Submit.

UDM Mapping

Log field	UDM mapping	Logic
@timestamp	metadata.event_timestamp	Converts the raw log field @timestamp to UDM Format.
category	security_result.rule_type	Maps the raw log field category to UDM.
operationName	metadata.product_event_type	Maps the raw log field operationName to UDM.
properties.Action	security_result.action	Maps the raw log field properties.Action to UDM, converting ALLOW to ALLOW, DENY to BLOCK, and any other value to UNKNOWN_ACTION.
properties.DestinationIp	target.ip	Maps the raw log field properties.DestinationIp to UDM.
properties.DestinationPort	target.port	Maps the raw log field properties.DestinationPort to UDM.
properties.DnssecOkBit	additional.fields.value.bool_value	Maps the raw log field properties.DnssecOkBit to UDM.
properties.EDNS0BufferSize	additional.fields.value.number_value	Maps the raw log field properties.EDNS0BufferSize to UDM.
properties.ErrorMessage	additional.fields.value.string_value	Maps the raw log field properties.ErrorMessage to UDM.
properties.ErrorNumber	additional.fields.value.number_value	Maps the raw log field properties.ErrorNumber to UDM.
properties.Policy	security_result.detection_fields.value	Maps the raw log field properties.Policy to UDM.
properties.Protocol	network.ip_protocol	Maps the raw log field properties.Protocol to UDM if it's not HTTPS or HTTP.
properties.Protocol	network.application_protocol	Maps the raw log field properties.Protocol to UDM if it's HTTPS or HTTP.
properties.QueryClass	network.dns.questions.class	Maps the raw log field properties.QueryClass to UDM using a lookup table for mapping DNS query classes.
properties.QueryId	network.dns.id	Maps the raw log field properties.QueryId to UDM.
properties.QueryName	network.dns.questions.name	Maps the raw log field properties.QueryName to UDM.
properties.QueryType	network.dns.questions.type	Maps the raw log field properties.QueryType to UDM using a lookup table for mapping DNS record types.
properties.RequestSize	network.sent_bytes	Maps the raw log field properties.RequestSize to UDM.
properties.ResponseCode	network.dns.response_code	Maps the raw log field properties.ResponseCode to UDM using a lookup table for mapping DNS response codes.
properties.ResponseFlags	additional.fields.value.string_value	Maps the raw log field properties.ResponseFlags to UDM.
properties.ResponseSize	network.received_bytes	Maps the raw log field properties.ResponseSize to UDM.
properties.Rule	security_result.rule_name	Maps the raw log field properties.Rule to UDM.
properties.RuleCollection	security_result.detection_fields.value	Maps the raw log field properties.RuleCollection to UDM.
properties.RuleCollectionGroup	security_result.detection_fields.value	Maps the raw log field properties.RuleCollectionGroup to UDM.
properties.SourceIp	principal.ip	Maps the raw log field properties.SourceIp to UDM.
properties.SourcePort	principal.port	Maps the raw log field properties.SourcePort to UDM.
properties.msg	security_result.description	Maps the raw log field properties.msg to UDM after extracting other fields from it.
records.category	security_result.rule_type	Maps the raw log field records.category to UDM.
records.operationName	metadata.product_event_type	Maps the raw log field records.operationName to UDM.
records.properties.msg		This field is used for extracting multiple fields using Grok patterns and doesn't have a direct mapping to UDM.
records.resourceId	metadata.product_log_id	Maps the raw log field records.resourceId to UDM.
resourceId	metadata.product_log_id	Maps the raw log field resourceId to UDM.
time	metadata.event_timestamp	Converts the raw log field time to UDM Format.
	metadata.vendor_name	This field is populated by the parser with the value Microsoft Inc..
	metadata.product_name	This field is populated by the parser with the value Azure Firewall Application Rule.
	metadata.log_type	This field is populated by the parser with the value AZURE_FIREWALL.
	additional.fields.key	This field is populated by the parser with the key for the additional field.
	security_result.detection_fields.key	This field is populated by the parser with the key for the detection field.
	network.application_protocol	This field is populated by the parser with the value DNS for DNS logs.
	metadata.event_type	This field is populated by the parser based on the log message. It can be NETWORK_CONNECTION, GENERIC_EVENT, STATUS_UPDATE, or NETWORK_DNS.

Changes

2024-12-26

Bug fix:

• When the logs contain the keyword Alert, mapped security_result.action to ALLOW.

2024-11-13

Enhancement:

• Mapped Action to security_result.detection_fields.
• Mapped Signature to security_result.detection_fields.
• Mapped IDS to security_result.detection_fields.
• Mapped Priority to security_result.priority_details.
• Mapped Classification to security_result.detection_fields.

2024-09-04

Enhancement:

• Mapped from_ip to dns.questions.name for DNS events.

2024-07-02

Enhancement:

• Added support to handle a new format of timestamp.

2024-04-29

Enhancement:

• Added support to handle a new format of ingested logs.

2024-02-07

Enhancement:

• Mapped ICMP type to additional.fields.
• Mapped Action and properties.Action to security_result.action_details.

2023-06-01

Enhancement:

• Newly ingested JSON logs of category AZFWDnsQuery are parsed.
• Mapped properties.msg in newly ingested JSON logs of category AzureFirewallNetworkRule.

2022-04-29

Bug fix:

• Newly ingested JSON logs are parsed to increase the overall parsing percentage.
• operationName mapped to metadata.product_event_type.
• resourceId mapped to metadata.product_log_id.

Need more help? Get answers from Community members and Google SecOps professionals.