Collect Microsoft Sentinel logs

Supported in:

Google SecOps SIEM

Note: This feature is covered by Pre-GA Offerings Terms of the Google Security Operations Service Specific Terms. Pre-GA features might have limited support, and changes to pre-GA features might not be compatible with other pre-GA versions. For more information, see the Google Security Operations Technical Support Service guidelines and the Google Security Operations Service Specific Terms.

Overview

This parser extracts fields from Microsoft Sentinel JSON logs, performs transformations like IP address extraction and string manipulation, and maps the extracted data to the UDM, including principal, target, security_result, and metadata fields. It also handles various data types and merges extracted entities into the UDM structure.

Note: This parser only ingests incidents and alerts.

Before you begin

• Ensure that you have a Google SecOps instance.
• Access to Microsoft Sentinel.

Configure a feed in Google SecOps to ingest the Microsoft Sentinel logs

1. Go to SIEM Settings > Feeds.
2. Click Add new.
3. In the Feed name field, enter a name for the feed (for example, Microsoft Sentinel Logs).
4. Select Webhook as the Source type.
5. Select Microsoft Sentinel as the Log type.
6. Click Next.
7. Optional: Specify values for the following input parameters:
   • Split delimiter: the delimiter that is used to separate log lines, such as \n.
   • Asset namespace: the asset namespace.
   • Ingestion labels: the label to be applied to the events from this feed.
8. Click Next.
9. Review your new feed configuration in the Finalize screen, and then click Submit.
10. Click Generate Secret Key to generate a secret key to authenticate this feed.
11. Copy and store the secret key as you cannot view this secret again. You can generate a new secret key again, but regeneration of the secret key makes the previous secret key obsolete.
12. From the Details tab, copy the feed endpoint URL from the Endpoint Information field. You need to specify this endpoint URL in your client application.
13. Click Done.

Create an API key for the webhook feed

1. Go to Google Cloud console > Credentials.

   Go to Credentials

2. Click Create credentials, and then select API key.

3. Restrict the API key access to the Google Security Operations API.

Specify the endpoint URL

1. In your client application, specify the HTTPS endpoint URL provided in the webhook feed.

2. Enable authentication by specifying the API key and secret key as part of the custom header in the following format:

   X-goog-api-key = API_KEY
   X-Webhook-Access-Key = SECRET
   

   Recommendation: Specify the API key as a header instead of specifying it in the URL. If your webhook client doesn't support custom headers, you can specify the API key and secret key using query parameters in the following format:

   ENDPOINT_URL?key=API_KEY&secret=SECRET
   

Replace the following:

• ENDPOINT_URL: the feed endpoint URL.
• API_KEY: the API key to authenticate to Google Security Operations.
• SECRET: the secret key that you generated to authenticate the feed.

Configure Logic App for Microsoft Sentinel Incidents

To configure Logic App for Microsoft Sentinel Incidents, follow these steps:

1. Sign in to Azure Portal.
2. Click Create a resource.
3. Search for Logic App.
4. Click Create to start the creation process.
5. Specify values for the following input parameters:
   • Subscription: Select the subscription.
   • Resource group: Select the resource group.
   • Name: Enter a name for the Logic App.
   • Region: Select the region.
   • Log Analytics workspace: Select the Log Analytics workspace.
6. Click Review + create.
7. Click Create.
8. After the Logic App is created, click Go to resource.
9. Click Development Tools > Logic App Designer.
10. Click Add a trigger.
11. Search for Microsoft Sentinel.
12. Select Microsoft Sentinel incident as the trigger.
13. If you haven't already created a connection to Microsoft Sentinel, you'll need to do so now. Click Create new and follow the prompts to authenticate.
14. Click Insert a new step.
15. Click Add an action.
16. Search for and select HTTP as the action.
17. Specify values for the following input parameters:
    • URI: the feed endpoint URL.
    • Method: POST
    • Headers: Add the following headers:
      • Content-Type: application/json
      • X-goog-api-key: the API key to authenticate to Google Security Operations.
      • X-Webhook-Access-Key: the secret key that you generated to authenticate the feed.

Configure Logic App for Microsoft Sentinel Alerts

To configure Logic App for Microsoft Sentinel Alerts, follow these steps:

1. Go to Azure Portal Home Page.
2. Click Create a resource.
3. Search for Logic App.
4. Click Create to start the creation process.
5. Specify values for the following input parameters:
   • Subscription: Select the subscription.
   • Resource group: Select the resource group.
   • Name: Enter a name for the Logic App.
   • Region: Select the region.
   • Log Analytics workspace: Select the Log Analytics workspace.
6. Click Review + create.
7. Click Create.
8. After the Logic App is created, click Go to resource.
9. Click Development Tools > Logic App Designer.
10. Click Add a trigger.
11. Search for Microsoft Sentinel.
12. Select Microsoft Sentinel alert as the trigger.
13. If you haven't already created a connection to Microsoft Sentinel, you'll need to do so now. Click Create new and follow the prompts to authenticate.
14. Click Insert a new step.
15. Click Add an action.
16. Search for and select HTTP as the action.
17. Specify values for the following input parameters:
    • URI: the feed endpoint URL.
    • Method: POST
    • Headers: Add the following headers:
      • Content-Type: application/json
      • X-goog-api-key: the API key to authenticate to Google Security Operations.
      • X-Webhook-Access-Key: the secret key that you generated to authenticate the feed.

Configure Automation rules for Microsoft Sentinel

To configure Automation rules for Microsoft Sentinel, follow these steps:

1. Go to your Microsoft Sentinel Workspace.
2. Click Configuration > Automation.
3. Click Create.
4. Select Automation rule.
5. Specify values for the following input parameters:
   • Name: Enter a name for the automation rule.
   • Trigger: select When incident is created.
   • Actions: select Run playbook > Logic App created for incidents.
6. Click Apply.
7. Click Create.
8. Select Automation rule.
9. Specify values for the following input parameters:
   • Name: Enter a name for the automation rule.
   • Trigger: select When incident is updated.
   • Condition: click Add > Condition (And) > Status > Changed.
   • Actions: select Run playbook > Logic App created for incidents.
10. Click Apply.
11. Click Create.
12. Select Automation rule.
13. Specify values for the following input parameters:
    • Name: Enter a name for the automation rule.
    • Trigger: select When alert is created.
    • Actions: select Run playbook > Logic App created for alerts.
14. Click Apply.

UDM Mapping Table

Log Field	UDM Mapping	Logic
AlertGenerationStatus	security_result.detection_fields.AlertGenerationStatus	Directly mapped from the ExtendedProperties field after JSON parsing.
AlertLink	principal.labels.AlertLink	Directly mapped.
AlertName	security_result.rule_name	Directly mapped.
AlertSeverity	security_result.severity	Directly mapped, converted to uppercase. If the value is one of HIGH, MEDIUM, LOW, CRITICAL, or UNKNOWN_SEVERITY, it's mapped to security_result.severity. Otherwise, it's mapped to security_result.severity_details.
AlertType	security_result.threat_name	Directly mapped.
Category	security_result.detection_fields.Category	Directly mapped from the ExtendedProperties field after JSON parsing.
CompromisedEntity	principal.resource.attribute.labels.CompromisedEntity	Directly mapped.
CompromisedEntityId	security_result.detection_fields.CompromisedEntityId	Directly mapped from the ExtendedProperties field after JSON parsing.
ConfidenceLevel	security_result.confidence_details	Directly mapped.
ConfidenceScore	security_result.detection_fields.ConfidenceScore	Directly mapped.
cribl_pipe	additional.fields.cribl_pipe	Directly mapped.
Description	security_result.description	Directly mapped.
DestinationDevice	security_result.detection_fields.DestinationDevice OR target.ip	Mapped from the ExtendedProperties field after JSON parsing. If the value is a valid IP address, it's mapped to target.ip. Otherwise, it's mapped as a detection field.
DestinationDeviceAddress	target.ip	Mapped from the ExtendedProperties field after JSON parsing, only if a valid IP address.
DeviceId	security_result.detection_fields.DeviceId	Directly mapped from the ExtendedProperties field after JSON parsing.
DisplayName	security_result.summary	Directly mapped.
EndTime	about.labels.EndTime	Directly mapped.
Entities.Address	principal.asset.ip	Extracted from the Entities array after JSON parsing. Only IP addresses are mapped.
Entities.HostName	principal.asset.hostname OR principal.asset.ip	Extracted from the Entities array after JSON parsing. If the value is a valid IP address, it's mapped to principal.asset.ip. Otherwise, it's mapped to principal.asset.hostname.
Entities.IoTDevice.DeviceId	security_result.detection_fields.IoTDeviceID	Extracted from the Entities array after JSON parsing.
Entities.IoTDevice.DeviceType	security_result.detection_fields.IoTDeviceType	Extracted from the Entities array after JSON parsing.
Entities.IoTDevice.DeviceTypeId	security_result.detection_fields.IoTDeviceTypeId	Extracted from the Entities array after JSON parsing.
Entities.IoTDevice.Importance	security_result.detection_fields.IoTDeviceImportance	Extracted from the Entities array after JSON parsing.
Entities.IoTDevice.IoTSecurityAgentId	security_result.detection_fields.IoTSecurityAgentId	Extracted from the Entities array after JSON parsing.
Entities.IoTDevice.Manufacturer	security_result.detection_fields.IoT Manufacturer	Extracted from the Entities array after JSON parsing.
Entities.IoTDevice.OperatingSystem	principal.asset.platform_software.platform_version	Extracted from the Entities array after JSON parsing, trailing spaces removed.
Entities.IoTDevice.PurdueLayer	security_result.detection_fields.IoT PurdueLayer	Extracted from the Entities array after JSON parsing.
Entities.IoTDevice.Sensor	security_result.detection_fields.IoT Sensor	Extracted from the Entities array after JSON parsing.
ExtendedProperties.Protocol	security_result.detection_fields.Protocol	Directly mapped from the ExtendedProperties field after JSON parsing.
ExtendedProperties.SensorId	security_result.detection_fields.SensorId	Directly mapped from the ExtendedProperties field after JSON parsing.
ExtendedProperties.SourceDevice	principal.ip OR security_result.detection_fields.SourceDevice	Mapped from the ExtendedProperties field after JSON parsing. If the value is a valid IP address, it's mapped to principal.ip. Otherwise, it's mapped as a detection field.
ExtendedProperties.SourceDeviceAddress	principal.ip	Mapped from the ExtendedProperties field after JSON parsing, only if a valid IP address.
IsIncident	security_result.detection_fields.IsIncident	Directly mapped, converted to string.
ProcessingEndTime	about.labels.ProcessingEndTime	Directly mapped.
ProductComponentName	principal.resource.attribute.labels.ProductComponentName	Directly mapped.
ProductName	principal.resource.attribute.labels.ProductName	Directly mapped.
ProviderName	principal.resource.attribute.labels.ProviderName	Directly mapped.
ResourceId	principal.resource.product_object_id, target.resource.name	Directly mapped.
SourceComputerId	principal.asset.asset_id	Directly mapped, prefixed with "SourceComputerId:".
SourceSystem	security_result.detection_fields.SourceSystem	Directly mapped.
StartTime	about.labels.StartTime	Directly mapped.
Status	security_result.detection_fields.Status	Directly mapped.
SystemAlertId	metadata.product_log_id	Directly mapped.
Tactics	security_result.attack_details.tactics.name	Extracted from the Tactics field after JSON parsing and backslash removal.
Techniques	security_result.attack_details.techniques.id	Extracted from the Techniques field after JSON parsing and backslash removal.
TenantId	additional.fields.TenantId	Directly mapped.
TimeGenerated	about.labels.TimeGenerated	Directly mapped.
timestamp	metadata.event_timestamp, events.timestamp	Directly mapped.
VendorName	metadata.vendor_name	Directly mapped.
VendorOriginalId	additional.fields.VendorOriginalId	Directly mapped.
_time	metadata.event_timestamp, events.timestamp	Parsed as a timestamp using UNIX or UNIX_MS format.
(Parser Logic)	metadata.event_type	Set to "USER_RESOURCE_ACCESS" if principal, target, and ResourceId are present. Otherwise, set to "GENERIC_EVENT".
(Parser Logic)	metadata.log_type	Set to "MICROSOFT_SENTINEL".
(Parser Logic)	metadata.product_name	Set to "MICROSOFT_SENTINEL".

Changes

2023-11-03

• Mapped "ResourceId" to "target.resource.name".
• When "ResourceId" is 'not null' and event has one of "principal" or "target" as 'not null', then map "metadata.event_type" to "USER_RESOURCE_ACCESS".

2023-08-31

• Newly created parser.