Collect Microsoft Windows AD data

This document includes the following information:

Deployment architecture and installation steps, plus any required configuration that produce logs supported by the Google Security Operations parser for Microsoft Windows Active Directory events. For an overview of Google Security Operations data ingestion, see Data ingestion to Google Security Operations.
Information about how the parser maps fields in the original log to Google Security Operations Unified Data Model fields.

Information in this document applies to the parser with the WINDOWS_AD ingestion label. The ingestion label identifies which parser normalizes raw log data to structured UDM format.

Before you begin

Review recommended deployment architecture

This diagram illustrates the recommended foundational components in a deployment architecture to collect and send Microsoft Windows events to Google Security Operations. Compare this information with your environment to be sure these components are installed. Each customer deployment will differ from this representation and may be more complex. The following is required:

All systems in the deployment architecture are configured with the UTC time zone.
Powershell script is created and configured on each Microsoft Windows AD server to collect USER_CONTEXT and ASSET_CONTEXT data.
NXLog is installed on each Microsoft Windows AD server to send data to the central Microsoft Windows or Linux server.
Google Security Operations forwarder is installed on the central Microsoft Windows or Linux server to forward log data to Google Security Operations.

Review supported devices and versions

The Google Security Operations Parser supports logs from the following Microsoft Windows server versions. Microsoft Windows Server is released with the following editions: Foundation, Essentials, Standard, and Datacenter. The event schema of logs generated by each edition does not differ.

Microsoft Windows Server 2019
Microsoft Windows Server 2016
Microsoft Windows Server 2012

Google Security Operations parser supports logs collected by NXLog Community or Enterprise Edition.

Review supported log types

Google Security Operations parser will parse and normalize data retrieved from the User Context and the Asset Context. It supports logs generated with English language text and is not supported with logs generated in non-English languages.

Configure Microsoft Windows AD servers

Steps to configure Microsoft Windows AD servers in the deployment architecture:

Configure all systems with the UTC time zone.
On each Microsoft Windows Active Directory server, create and configure a Powershell script to collect log data to an output file. NXLog reads the output file and sends data to the central Microsoft Windows or Linux server.

Create the Powershell script. See the following example. Change the value of $OUTPUT_FILENAME to the location where the output file should be written. This file will be read by NXLog. Data must be stored in JSON format. Set encoding to utf8. Use the -Filter parameter, rather than the -LDAPFilter parameter when calling the Get-ADUser and Get-ADComputer cmdlets.

# Set the location where the log file will be written
$OUTPUT_FILENAME="<Path_of_the_output_file>"

If (Test-Path -Path $OUTPUT_FILENAME) { Remove-Item -path $OUTPUT_FILENAME -ErrorAction SilentlyContinue}

# USER_CONTEXT: Gets all Active Directory users and their properties.
Get-ADUser -Filter * -properties samAccountName | % { Get-ADUser $_.SamAccountName -properties * | ConvertTo-JSON -compress | Out-File -encoding utf8 $OUTPUT_FILENAME -Append }

# ASSET_CONTEXT: Gets all Active Directory assets and their properties.
Get-ADComputer -Filter * -properties samAccountName | % { Get-ADComputer $_.SamAccountName -properties * | ConvertTo-JSON -compress | Out-File -encoding utf8 $OUTPUT_FILENAME -Append }

Create a recurring task that runs the script to fetch and write data to the output file.
1. Open the Task Scheduler application.
2. Click Create task.
3. Enter a name and description for the task.
4. Select the Run with highest privileges checkbox to make sure all data is retrieved.
5. In the Triggers tab, define when you want to repeat the task.
6. In the Action tab, add a new action and provide the path of the file where the script is stored.
Install the NXLog Agent on each Microsoft Windows Active Directory server. This application forwards logs to the central Microsoft Windows or Linux server. Follow the NXLog documentation.

Create a configuration file for each NXLog instance. Use the NXLog im_file module to read from the file and parse the lines into fields. Use the om_tcp module to forward data to the central Microsoft Windows or Linux server.

Here is an example NXLog configuration. Replace <hostname> and <port> values with information about the destination central Microsoft Windows or Linux Server. In the <Input in_adcontext> section and File property, add the path of the output log file written by the Powershell script. Always set DirCheckInterval and PollInterval. If these are not defined, NXLog polls for files every 1 second.

define ROOT C:\Program Files\nxlog
define ADCONTEXT_OUTPUT_DESTINATION_ADDRESS <hostname>
define ADCONTEXT_OUTPUT_DESTINATION_PORT <port>

Moduledir   %ROOT%\modules
CacheDir    %ROOT%\data
Pidfile     %ROOT%\data\nxlog.pid
SpoolDir    %ROOT%\data
LogFile     %ROOT%\data\nxlog.log

<Input in_adcontext>
    Module im_file
    File "<Path_of_the_output_file>"
    DirCheckInterval 3600
    PollInterval 3600
</Input>

<Output out_chronicle_adcontext>
    Module  om_tcp
    Host    %ADCONTEXT_OUTPUT_DESTINATION_ADDRESS%
    Port    %ADCONTEXT_OUTPUT_DESTINATION_PORT%
</Output>

<Route ad_context_to_chronicle>
    Path in_adcontext => out_chronicle_adcontext
</Route>

Start the NXLog service in each system.

Configure the central Microsoft Windows or Linux server

See the Installing and configuring the forwarder on Linux or Installing and configuring the forwarder on Microsoft Windows for information about installing and configuring the forwarder.

Configure the system with the UTC time zone.
Install the Google Security Operations forwarder on the central Microsoft Windows or Linux server.

Configure the Google Security Operations forwarder to send logs to Google Security Operations. Here is an example Forwarder configuration.

  - syslog:
      common:
        enabled: true
        data_type: WINDOWS_AD
        batch_n_seconds: 10
        batch_n_bytes: 1048576
      tcp_address: 0.0.0.0:10518
      connection_timeout_sec: 60

Field mapping reference: device log fields to UDM fields

This section describes how the parser maps original log fields to Unified Data Model fields.

User Context logs

NXLog field	UDM field
`GivenName`	`entity.entity.user.first_name`
`Surname`	`entity.entity.user.last_name`
`SamAccountName`	`entity.entity.user.userid`
`SID.Value`	`entity.entity.user.windows_sid`
`ObjectClass`	If the value is `user`, `entity.metadata.entity_type` is set to `USER`
`ObjectGuid`	If the `ObjectGuid` log field value is not empty, then the `ObjectGuid` log field is mapped to the `entity.entity.user.product_object_id` UDM field. Else, If the `objectGUID` log field value is not empty, then the `objectGUID` log field is mapped to the `entity.entity.user.product_object_id` UDM field. Else, If the `ObjectGUID` log field value is not empty, then the `ObjectGUID` log field is mapped to the `entity.entity.user.product_object_id` UDM field.
`objectGUID`	If the `ObjectGuid` log field value is not empty, then the `ObjectGuid` log field is mapped to the `entity.entity.user.product_object_id` UDM field. Else, If the `objectGUID` log field value is not empty, then the `objectGUID` log field is mapped to the `entity.entity.user.product_object_id` UDM field. Else, If the `ObjectGUID` log field value is not empty, then the `ObjectGUID` log field is mapped to the `entity.entity.user.product_object_id` UDM field.
`ObjectGUID`	If the `ObjectGuid` log field value is not empty, then the `ObjectGuid` log field is mapped to the `entity.entity.user.product_object_id` UDM field. Else, If the `objectGUID` log field value is not empty, then the `objectGUID` log field is mapped to the `entity.entity.user.product_object_id` UDM field. Else, If the `ObjectGUID` log field value is not empty, then the `ObjectGUID` log field is mapped to the `entity.entity.user.product_object_id` UDM field.
`AccountExpirationDate`	`entity.entity.user.account_expiration_time`
`badPwdCount`	`entity.entity.user.attribute.label.key` and `entity.entity.user.attribute.label.value`
`CanonicalName`	`entity.entity.administrative_domain`
`City`	`entity.entity.user.personal_address.city`
`Company`	`entity.entity.user.company_name`
`Country`	`entity.entity.user.personal_address.country_or_region`
`Department`	`entity.entity.user.department`
`Description`	`entity.metadata.description`
`DisplayName`	`entity.entity.user.user_display_name`
`EmailAddress`	`entity.entity.user.email_addresses`
`EmployeeID`	`entity.entity.user.employee_id`
`HomeDirectory`	`entity.entity.file.full_path`
`HomePage`	`entity.entity.url`
`HomePhone`	`entity.entity.user.phone_numbers`
`LastBadPasswordAttempt`	`entity.entity.user.last_bad_password_attempt_time`
`lastLogoff`	`entity.entity.user.attribute.label.key` and `entity.entity.user.attribute.label.value`
`lastLogon`	`entity.entity.user.attribute.label.key` and `entity.entity.user.attribute.label.value`
`LastLogonDate`	`entity.entity.user.last_login_time`
`Manager`	Values for `GUID`, `SAMAccountname`, `SID` all mapped to different UDM fields: - `SID` is stored in manager.windows_sid - `Distinguished` name (i.e. value in first CN) is stored in `manager.user_display_name` - `GUID`,`SamAccountName` is stored in `manager.userid`
`MemberOf`	The following fields in the first occurrence of `CN` are set: `entity.relations.entity.group.group_display_name` `entity.relations.entity_type` set to `GROUP` `entity.relations.relationship` set t0 `MEMBER` `entity.relations.direction` set to `UNIDIRECTIONAL`
`MobilePhone`	`entity.entity.user.phone_numbers`
`Office`	`entity.entity.user.office_address.name`
`PasswordExpired`	`entity.entity.user.attribute.label.key` and `entity.entity.user.attribute.label.value`
`PasswordLastSet`	`entity.entity.user.last_password_change_time`
`PasswordNeverExpires`	`entity.entity.user.attribute.label.key` and `entity.entity.user.attribute.label.value`
`PasswordNotRequired`	`entity.entity.user.attribute.label.key` and `entity.entity.user.attribute.label.value`
`PrimaryGroup`	Following fields are set: - `entity.relations.entity.group.group_display_name` - `entity.relations.entity_type` set to `GROUP` - `entity.relations.relationship` set to `MEMBER` - `entity.relations.direction` set to `UNIDIRECTIONAL`
`ServicePrincipalNames`	`entity.entity.user.attribute.label.key` and `entity.entity.user.attribute.label.value`
`State`	`entity.entity.user.personal_address.state`
`StreetAddress`	`entity.entity.user.personal_address.name`
`Title`	`entity.entity.user.title`
`whenCreated`	`entity.user.attribute.creation_time`
`AccountLockoutTime`	`entity.entity.user.account_lockout_time`

Asset Context logs

NXLog Field	UDM Field
`DNSHostName`	`entity.entity.asset.hostname`
`SamAccountName`	`entity.entity.asset.asset_id`
`SID.Value`	`entity.entity.user.windows_sid`
`ObjectClass`	If the value is `computer`, `entity.metadata.entity_type` set to `ASSET`
`ObjectGuid`	If the `ObjectGuid` log field value is not empty, then the `ObjectGuid` log field is mapped to the `entity.entity.asset.product_object_id` UDM field. Else, If the `ObjectGUID` log field value is not empty, then the `ObjectGUID` log field is mapped to the `entity.entity.asset.product_object_id` UDM field.
`ObjectGUID`	If the `ObjectGuid` log field value is not empty, then the `ObjectGuid` log field is mapped to the `entity.entity.asset.product_object_id` UDM field. Else, If the `ObjectGUID` log field value is not empty, then the `ObjectGUID` log field is mapped to the `entity.entity.asset.product_object_id` UDM field.
`AccountExpirationDate`	`entity.entity.user.account_expiration_time`
`badPwdCount`	`entity.entity.asset.attribute.label.key` and `entity.entity.asset.attribute.label.value`
`CanonicalName`	`entity.entity.administrative_domain`
`countryCode`	`entity.entity.asset.location.country_or_region`
`Description`	`entity.entity.metadata.description`
`HomePage`	`entity.entity.url`
`IPv4Address`	`entity.entity.asset.ip`
`IPv6Address`	`entity.entity.asset.ip`
`LastBadPasswordAttempt`	`entity.entity.user.last_bad_password_attempt_time`
`lastLogoff`	`entity.entity.asset.attribute.label.key` and `entity.entity.asset.attribute.label.value`
`lastLogon`	`entity.entity.asset.attribute.label.key` and `entity.entity.asset.attribute.label.value`
`LastLogonDate`	`entity.entity.user.last_login_time`
`Location`	`entity.entity.asset.location.name`
`ManagedBy`	The following fields are set: `entity.entity.user.user_display_name` `entity.relations.entity_type` set to `USER` `entity.relations.relationship` set to `ADMINISTERS` `entity.relations.direction` set to `UNIDIRECTIONAL`
`ObjectCategory`	`entity.entity.asset.category`
`OperatingSystem`	If the `name` contains `Windows`, `entity.entity.asset.platform_software.platform` field is set to `WINDOWS`.
`OperatingSystemServicePack`	`entity.entity.asset.platform_software.platform_patch_level`
`OperatingSystemVersion`	The field `entity.entity.asset.platform_software.platform_version` is set to `%{OperatingSystem} - %{OperatingSystemVersion}`
`PasswordExpired`	`entity.entity.asset.attribute.label.key` and `entity.entity.asset.attribute.label.value`
`PasswordLastSet`	`entity.entity.user.last_password_change_time`
`PasswordNeverExpires`	`entity.entity.asset.attribute.label.key` and `entity.entity.asset.attribute.label.value`
`PasswordNotRequired`	`entity.entity.asset.attribute.label.key` and `entity.entity.asset.attribute.label.value`
`PrimaryGroup`	The following fields are set: - `entity.relations.entity.group.group_display_name` - `entity.relations.entity_type` set to `GROUP` - `entity.relations.relationship` set to `MEMBER` - `entity.relations.direction` set to `UNIDIRECTIONAL`
`ServicePrincipalNames`	`entity.entity.asset.attribute.label.key` and `entity.entity.asset.attribute.label.value`
`whenChanged`	`entity.entity.asset.attribute.last_update_time`
`whenCreated`	`entity.entity.asset.attribute.creation_time`
`AccountLockoutTime`	`entity.entity.user.account_lockout_time`