Collect Azure APP Service logs

Supported in:

Google SecOps SIEM

Note: This feature is covered by Pre-GA Offerings Terms of the Google Security Operations Service Specific Terms. Pre-GA features might have limited support, and changes to pre-GA features might not be compatible with other pre-GA versions. For more information, see the Google Security Operations Technical Support Service guidelines and the Google Security Operations Service Specific Terms.

This document explains how to export Azure APP Service logs to Google Security Operations using an Azure Storage Account. The parser transforms raw JSON formatted Azure App Service logs into a structured Unified Data Model (UDM). It extracts relevant fields from the raw logs, performs data cleaning and normalization, and maps the extracted information to corresponding UDM fields, ultimately outputting a UDM-compliant JSON object for each log entry.

Before you begin

• Ensure that you have A Google SecOps instance.
• Ensure that you have an active Azure tenant.
• Ensure that you have privileged access to Azure.

Configure Azure Storage Account

1. In the Azure console, search for Storage accounts.
2. Click + Create.
3. Specify values for the following input parameters:
   • Subscription: select the subscription.
   • Resource Group: select the resource group.
   • Region: select the region.
   • Performance: select the performance (Standard recommended).
   • Redundancy: select the redundancy (GRS or LRS recommended).
   • Storage account name: enter a name for the new storage account.
4. Click Review + create.
5. Review the overview of the account and click Create.
6. From the Storage Account Overview page, select the Access keys submenu in Security + networking.
7. Click Show next to key1 or key2
8. Click Copy to clipboard to copy the key.
9. Save the key in a secure location for later use.
10. From the Storage Account Overview page, select the Endpoints submenu in Settings.
11. Click Copy to clipboard to copy the Blob service endpoint URL (for example, https://<storageaccountname>.blob.core.windows.net)
12. Save the endpoint URL in a secure location for later use.

Configure Log Export for Azure AD Sign-In Logs

1. Sign in to the Azure Portal using you privileged account.
2. Go to App Services and select the required app service in use.
3. Select Monitoring > App Service Logs.
4. Turn ON for Application Logging (blob).
5. Select Storage under Web Service Logging.
6. Select the Subscription and Storage Account.
7. Define the Retention Period and Quota according to your requirements.
8. Turn ON for Detailed error messages.
9. Turn ON for Failed request tracing.
10. Click Save.

Configure a feed in Google SecOps to ingest the Azure APP Service logs

1. Go to SIEM Settings > Feeds.
2. Click Add new.
3. In the Feed name field, enter a name for the feed (for example, Azure APP Service Logs).
4. Select Microsoft Azure Blob Storage as the Source type.
5. Select Azure APP Service as the Log type.
6. Click Next.

7. Specify values for the following input parameters:

   • Azure URI: the blob endpoint URL.
     • ENDPOINT_URL/BLOB_NAME
       • Replace the following:
       • ENDPOINT_URL: the blob endpoint URL (https://<storageaccountname>.blob.core.windows.net)
       • BLOB_NAME: the name of the blob (such as, <logname>-logs)
   • URI is a: select the URI TYPE according to log stream configuration (Single file | Directory | Directory which includes subdirectories).

   • Source deletion options: select the deletion option according to your preference.

     Note: If you select the Delete transferred files or Delete transferred files and empty directories option, make sure that you granted appropriate permissions to the service account.

   • Shared key: the access key to the Azure Blob Storage.

   • Asset namespace: the asset namespace.

   • Ingestion labels: the label to be applied to the events from this feed.

8. Click Next.

9. Review your new feed configuration in the Finalize screen, and then click Submit.

UDM Mapping Table

Log Field	UDM Mapping	Logic
AppRoleInstance	read_only_udm.principal.resource.product_object_id	Direct mapping
AppRoleName	read_only_udm.principal.resource.name	Direct mapping
AppVersion	read_only_udm.principal.resource.attribute.labels.value	Direct mapping
Category	read_only_udm.metadata.product_event_type	Direct mapping
CIp	read_only_udm.target.asset.ip	Direct mapping
CIp	read_only_udm.target.ip	Direct mapping
ClientCity	read_only_udm.principal.location.city	Direct mapping
ClientCountryOrRegion	read_only_udm.principal.location.country_or_region	Direct mapping
ClientIP	read_only_udm.principal.asset.ip	Direct mapping
ClientIP	read_only_udm.principal.ip	Direct mapping
ClientStateOrProvince	read_only_udm.principal.location.state	Direct mapping
ClientType	read_only_udm.additional.fields.value.string_value	Direct mapping
ComputerName	read_only_udm.principal.asset.hostname	Direct mapping
ComputerName	read_only_udm.principal.hostname	Direct mapping
Cookie	read_only_udm.principal.resource.attribute.labels.value	Direct mapping
CsBytes	read_only_udm.network.sent_bytes	Renamed from CsBytes
CsHost	read_only_udm.additional.fields.value.string_value	Direct mapping
CsMethod	read_only_udm.network.http.method	Direct mapping
CsUriQuery	read_only_udm.principal.resource.attribute.labels.value	Direct mapping
CsUriStem	read_only_udm.additional.fields.value.string_value	Direct mapping
CsUriStem	read_only_udm.target.url	Direct mapping
CsUsername	read_only_udm.principal.user.user_display_name	Direct mapping
EventIpAddress	read_only_udm.principal.asset.ip	Direct mapping
EventIpAddress	read_only_udm.principal.ip	Direct mapping
EventPrimaryStampName	read_only_udm.additional.fields.value.string_value	Direct mapping
EventStampName	read_only_udm.additional.fields.value.string_value	Direct mapping
EventStampType	read_only_udm.additional.fields.value.string_value	Direct mapping
Host	read_only_udm.principal.asset.hostname	Direct mapping
Host	read_only_udm.principal.hostname	Direct mapping
IKey	read_only_udm.target.resource.attribute.labels.value	Direct mapping
Instance	read_only_udm.additional.fields.value.string_value	Direct mapping
Name	read_only_udm.additional.fields.value.string_value	Direct mapping
Protocol	read_only_udm.additional.fields.value.string_value	Direct mapping
Protocol	read_only_udm.network.application_protocol	Mapped to HTTP if Protocol is HTTP/1.1
Referer	read_only_udm.network.http.referral_url	Direct mapping
ResourceGUID	read_only_udm.target.resource.product_object_id	Renamed from ResourceGUID
SDKVersion	read_only_udm.additional.fields.value.string_value	Direct mapping
SDKVersion	read_only_udm.principal.resource.attribute.labels.value	Direct mapping
SPort	read_only_udm.principal.port	Renamed from SPort
ScBytes	read_only_udm.network.received_bytes	Renamed from ScBytes
ScStatus	read_only_udm.network.http.response_code	Renamed from ScStatus
TimeTaken	read_only_udm.additional.fields.value.string_value	Direct mapping
Type	read_only_udm.additional.fields.value.string_value	Direct mapping
User	read_only_udm.principal.user.userid	Direct mapping
UserAddress	read_only_udm.principal.asset.ip	Extracted from UserAddress if it's a valid IP address
UserAddress	read_only_udm.principal.ip	Extracted from UserAddress if it's a valid IP address
UserAgent	read_only_udm.network.http.user_agent	Direct mapping
UserDisplayName	read_only_udm.principal.user.user_display_name	Direct mapping
category	read_only_udm.metadata.product_event_type	Direct mapping
level	read_only_udm.security_result.severity	Uppercased and renamed from level
location	read_only_udm.principal.location.name	Direct mapping
operationName	read_only_udm.additional.fields.value.string_value	Direct mapping
record.properties.Protocol	read_only_udm.additional.fields.value.string_value	Direct mapping
record.properties.Result	read_only_udm.security_result.summary	Direct mapping
record.time	read_only_udm.metadata.event_timestamp	Parsed as RFC 3339 timestamp
resourceId	read_only_udm.target.resource.attribute.labels.value	Direct mapping
resourceId	read_only_udm.target.resource.product_object_id	Renamed from resourceId
	read_only_udm.metadata.event_type	Determined based on the presence of principal, target, and Protocol. Set to NETWORK_HTTP if principal, target, and Protocol=HTTP are present. Set to NETWORK_CONNECTION if principal and target are present. Set to STATUS_UPDATE if only principal is present. Otherwise, set to GENERIC_EVENT.

Changes

2024-10-18

Enhancement:

• Mapped properties.XForwardedHost to principal.hostname and principal.asset.hostname.
• Mapped properties_category to additional.fields.
• Mapped properties.roleInstance to principal.resource.product_object_id.
• Mapped properties.message to security_result.summary.

2024-09-30

Enhancement:

• Added support to parse the new format of unparsed JSON logs.

2024-06-24

Enhancement:

• Mapped metadata.product_name and metadata.vendor_name to AZURE_APP_SERVICE.

2024-06-07

Enhancement:

• Added support for JSON logs, when they are not in an array format.
• Mapped properties.ScSubStatus to additional.fields.
• Mapped properties.ScWin32Status to additional.fields.

2024-04-25

Enhancement:

• Mapped properties.User to principal.user.userid.
• Mapped properties.UserDisplayName to principal.user.user_display_name.
• Mapped properties.UserAddress to principal.ip.
• Mapped properties.Protocol, ClientBrowser, ClientModel, ClientOS, OperationId, ParentId, and ItemCount to additional.fields.
• Mapped CsUriQuery, SDKVersion, and Cookie to principal.resource.attribute.labels.
• Mapped SessionId to network.session_id.
• Mapped Message to security_result.summary.
• Mapped SeverityLevel to security_result.severity_details.

2024-02-20

Enhancement:

• Mapped record.AppRoleInstance to principal.resource.product_object_id.
• Mapped record.AppRoleName to principal.resource.name.
• Mapped record.ClientCity to principal.location.city.
• Mapped record.ClientCountryOrRegion to principal.location.country_or_region.
• Mapped record.ClientStateOrProvince to principal.location.state.
• Mapped record.ClientIP to principal.ip.
• Mapped Result to security_result.summary.
• Mapped UserAgent to network.http.user_agent.
• Mapped Referer to network.http.referral_url.
• Mapped record.ResourceGUID to target.resource.product_object_id.
• If record.ResourceGUID is not present, then mapped record.resourceId to target.resource.product_object_id.
• If record.ResourceGUID is present, then mapped record.resourceId to additional.fields.
• Mapped record.Type to additional.fields.
• Mapped record.ClientType to additional.fields.
• Mapped record.SDKVersion to additional.fields.
• Mapped record.Name to additional.fields.
• Mapped record.Instance to additional.fields.
• Mapped record.TimeTaken to additional.fields.
• Mapped record.Cookie to additional.fields.
• Mapped record.AppVersion to principal.resource.attribute.labels.
• Mapped record.IKey to target.resource.attribute.labels.
• Mapped record.Category to metadata.product_event_type.
• Mapped CsUriStem to target.url.
• Changed mapping of CsBytes from network.received_bytes to network.sent_bytes.
• Changed mapping of ScBytes from network.sent_bytes to network.received_bytes.

2023-12-07

• Newly created parser.

Need more help? Get answers from Community members and Google SecOps professionals.