Collect Tanium Stream logs

Supported in:

Google secops SIEM

Note: This feature is covered by Pre-GA Offerings Terms of the Google Security Operations Service Specific Terms. Pre-GA features might have limited support, and changes to pre-GA features might not be compatible with other pre-GA versions. For more information, see the Google SecOps Technical Support Service guidelines and the Google SecOps Service Specific Terms.

This document explains how to ingest Tanium Stream logs to Google Security Operations using Tanium Connect's native AWS S3 export functionality. Tanium Stream provides real-time endpoint telemetry, threat hunting data, and behavioral analytics in JSON format, which can be directly exported to S3 using Tanium Connect without requiring custom Lambda functions. The parser transforms raw JSON logs from Tanium Stream into a unified data model (UDM). It first normalizes common fields and then applies specific logic based on the "logType" or "eventType" to map relevant information into the appropriate UDM fields, handling various event types like network connections, user logins, process launches, and file modifications.

Before you begin

Make sure you have the following prerequisites:

• A Google SecOps instance
• Tanium Core Platform 7.0 or later
• Tanium Stream module installed and configured
• Tanium Connect module installed with valid license
• Tanium Threat Response 3.4.346 or later (if using TR integration)
• Privileged access to Tanium Console with administrative rights
• Privileged access to AWS (S3, IAM)

Configure Tanium Stream service account

1. Sign in to the Tanium Console.
2. Go to Modules > Stream.
3. Click Settings at the top right.
4. In the Service Account section, configure the following:
   • Service Account User: Select a user with appropriate Stream permissions.
   • Verify the account has Connect User role privilege.
   • Confirm access to Stream data sources and endpoints.
5. Click Save to apply the service account configuration.

Collect Tanium Stream prerequisites

1. Sign in to the Tanium Console as an administrator.
2. Go to Administration > Permissions > Users.
3. Create or identify a service account user with the following roles:
   • Stream Administrator or Stream Read Only User role.
   • Connect User role privilege.
   • Access to monitored computer groups (recommended: All Computers group).
   • Read Saved Question permission for Stream content sets.

Note: The service account credentials are used internally by Tanium Connect to authenticate and retrieve Stream telemetry data. These credentials are configured during the Tanium Connect source setup within the Tanium Console interface.

Configure AWS S3 bucket and IAM for Google SecOps

1. Create Amazon S3 bucket following this user guide: Creating a bucket
2. Save bucket Name and Region for future reference (for example, tanium-stream-logs).
3. Create a user following this user guide: Creating an IAM user.
4. Select the created User.
5. Select the Security credentials tab.
6. Click Create Access Key in the Access Keys section.
7. Select Third-party service as the Use case.
8. Click Next.
9. Optional: add a description tag.
10. Click Create access key.
11. Click Download CSV file to save the Access Key and Secret Access Key for later use.
12. Click Done.
13. Select the Permissions tab.
14. Click Add permissions in the Permissions policies section.
15. Select Add permissions.
16. Select Attach policies directly
17. Search for and select the AmazonS3FullAccess policy.
18. Click Next.
19. Click Add permissions.

Configure Tanium Connect AWS S3 destination

1. Sign in to the Tanium Console.
2. Go to Modules > Connect.
3. Click Create Connection.
4. Provide the following configuration details:
   • Name: Enter a descriptive name (for example, Stream Telemetry to S3 for SecOps).
   • Description: Optional description (for example, Export endpoint telemetry and threat hunting data to AWS S3 for Google SecOps ingestion).
   • Enable: Select to enable the connection to run on schedule.
5. Click Next.

Configure the connection source

1. In the Source section, provide the following configuration details:
   • Source Type: Select Saved Question.
   • Saved Question: Select one of the following Stream-related saved questions:
     • Stream - Endpoint Events for real-time endpoint telemetry.
     • Stream - Network Events for network activity monitoring.
     • Stream - Process Events for process execution tracking.
     • Stream - File Events for file system activity.
     • Stream - Threat Hunting Data for behavioral analytics.
   • Computer Group: Select All Computers or specific computer groups to monitor.
   • Refresh Interval: Set appropriate interval for data collection (for example, 5 minutes for real-time telemetry).
2. Click Next.

Configure AWS S3 destination

1. In the Destination section, provide the following configuration details:
   • Destination Type: Select AWS S3.
   • Destination Name: Enter a unique name (for example, Google SecOps Stream S3 Destination).
   • AWS Access Key: Enter the AWS access key from the CSV file downloaded in the AWS S3 configuration step.
   • AWS Secret Access Key: Enter the AWS secret access key from the CSV file downloaded in the AWS S3 configuration step.
   • Bucket Name: Enter your S3 bucket name (for example, tanium-stream-logs).
   • Region: Select the AWS region where your S3 bucket is located.
   • Key Prefix: Enter a prefix for the S3 objects (for example, tanium/stream/).
2. Click Next.

Configure filters

1. In the Filters section, configure data filtering options:
   • Send new items only: Select this option to send only new telemetry data since the last export.
   • Column filters: Add filters based on specific event attributes if needed (for example, filter by event type, process name, or threat indicators).
2. Click Next.

Format data for AWS S3

1. In the Format section, configure the data format:
   • Format: Select JSON.
   • Options:
     • Include headers: Deselect to avoid headers in JSON output.
     • Include empty cells: Select based on your preference.
   • Advanced Options:
     • File naming: Use default timestamp-based naming.
     • Compression: Select Gzip to reduce storage costs and transfer time.
2. Click Next.

Schedule the connection

1. In the Schedule section, configure the export schedule:
   • Enable schedule: Select to enable automatic scheduled exports.
   • Schedule type: Select Recurring.
   • Frequency: Select Every 5 minutes for near real-time telemetry data.
   • Start time: Set appropriate start time for the first export.
2. Click Next.

Save and verify connection

1. Review the connection configuration in the summary screen.
2. Click Save to create the connection.
3. Click Test Connection to verify the configuration.
4. If the test is successful, click Run Now to perform an initial export.
5. Monitor the connection status in the Connect Overview page.

Configure a feed in Google SecOps to ingest Tanium Stream logs

1. Go to SIEM Settings > Feeds.
2. Click + Add New Feed.
3. In the Feed name field, enter a name for the feed (for example, Tanium Stream logs).
4. Select Amazon S3 V2 as the Source type.
5. Select Tanium Stream as the Log type.
6. Click Next.
7. Specify values for the following input parameters:
   • S3 URI: s3://tanium-stream-logs/tanium/stream/
   • Source deletion options: Select deletion option according to your preference.
   • Maximum File Age: Include files modified in the last number of days. Default is 180 days.
   • Access Key ID: User access key with access to the S3 bucket.
   • Secret Access Key: User secret key with access to the S3 bucket.
   • Asset namespace: The asset namespace.
   • Ingestion labels: The label applied to the events from this feed.
8. Click Next.
9. Review your new feed configuration in the Finalize screen, and then click Submit.

UDM Mapping Table

Log field	UDM mapping	Logic
additional.event__AlgorithmName	additional.fields.key: eventAlgorithmName additional.fields.value.string_value: %{additional.eventAlgorithmName}	Value taken directly from the raw log field additional.event__AlgorithmName.
additional.event__AuthenticationPackageName	target.resource.name: %{additional.event__AuthenticationPackageName}	Value taken directly from the raw log field additional.event__AuthenticationPackageName.
additional.event__CallerProcessId	principal.process.pid: %{additional.event__CallerProcessId}	Value taken directly from the raw log field additional.event__CallerProcessId.
additional.event__CallerProcessName	principal.process.file.fullpath: %{additional.event_CallerProcessName}	Value taken directly from the raw log field additional.event__CallerProcessName.
additional.event__ClientProcessId	principal.process.pid: %{additional.event__ClientProcessId}	Value taken directly from the raw log field additional.event__ClientProcessId.
additional.event__ClientProcessStartKey	additional.fields.key: eventClientProcessStartKey additional.fields.value.string_value: %{additional.eventClientProcessStartKey}	Value taken directly from the raw log field additional.event__ClientProcessStartKey.
additional.event__CommandLine	target.process.commandline: %{additional.event_CommandLine}	Value taken directly from the raw log field additional.event__CommandLine.
additional.event__ElevatedToken	additional.fields.key: event__ElevatedToken additional.fields.value.string_value: Yes/No	Value taken directly from the raw log field additional.event__ElevatedToken. If the value is "%%1842", it is replaced with "Yes". If the value is "%%1843", it is replaced with "No".
additional.event__FQDN	principal.hostname: %{additional.event__FQDN}	Value taken directly from the raw log field additional.event__FQDN.
additional.event__FailureReason	additional.fields.key: eventFailureReason additional.fields.value.string_value: %{additional.eventFailureReason}	Value taken directly from the raw log field additional.event__FailureReason.
additional.event__ImpersonationLevel	additional.fields.key: eventImpersonationLevel additional.fields.value.string_value: %{additional.eventImpersonationLevel}	Value taken directly from the raw log field additional.event__ImpersonationLevel.
additional.event__IpAddress	target.ip: %{additional.event__IpAddress}	Value taken directly from the raw log field additional.event__IpAddress.
additional.event__IpPort	target.port: %{additional.event__IpPort}	Value taken directly from the raw log field additional.event__IpPort and converted to an integer.
additional.event__KeyLength	additional.fields.key: eventKeyLength additional.fields.value.string_value: %{additional.eventKeyLength}	Value taken directly from the raw log field additional.event__KeyLength.
additional.event__KeyName	additional.fields.key: eventKeyName additional.fields.value.string_value: %{additional.eventKeyName}	Value taken directly from the raw log field additional.event__KeyName.
additional.event__KeyType	additional.fields.key: eventKeyType additional.fields.value.string_value: %{additional.eventKeyType}	Value taken directly from the raw log field additional.event__KeyType.
additional.event__LmPackageName	additional.fields.key: eventLmPackageName additional.fields.value.string_value: %{additional.eventLmPackageName}	Value taken directly from the raw log field additional.event__LmPackageName.
additional.event__LogonGuid	target.resource.product_objectid: %{additional.event_LogonGuid}	Value taken directly from the raw log field additional.event__LogonGuid with curly braces removed.
additional.event__LogonProcessName	target.process.file.fullpath: %{additional.event_LogonProcessName}	Value taken directly from the raw log field additional.event__LogonProcessName.
additional.event__LogonType	extensions.auth.authdetails: Logon Type: %{additional.event_LogonType}	Value taken directly from the raw log field additional.event__LogonType.
additional.event__MandatoryLabel	additional.fields.key: eventMandatoryLabel additional.fields.value.string_value: %{additional.eventMandatoryLabel}	Value taken directly from the raw log field additional.event__MandatoryLabel.
additional.event__NewProcessId	target.process.pid: %{additional.event__NewProcessId}	Value taken directly from the raw log field additional.event__NewProcessId.
additional.event__NewProcessName	target.process.file.fullpath: %{additional.event_NewProcessName}	Value taken directly from the raw log field additional.event__NewProcessName.
additional.event__ObjectServer	security_result.categorydetails: %{additional.event_ObjectServer}	Value taken directly from the raw log field additional.event__ObjectServer.
additional.event__Operation	additional.fields.key: eventOperation additional.fields.value.string_value: %{additional.eventOperation}	Value taken directly from the raw log field additional.event__Operation.
additional.event__ParentProcessId	principal.process.parentprocess.pid: %{additional.event_ParentProcessId}	Value taken directly from the raw log field additional.event__ParentProcessId.
additional.event__ParentProcessName	principal.process.parent_process.file.fullpath: %{additional.event_ParentProcessName}	Value taken directly from the raw log field additional.event__ParentProcessName.
additional.event__ProcessId	principal.process.pid: %{additional.event__ProcessId}	Value taken directly from the raw log field additional.event__ProcessId.
additional.event__ProcessName	principal.process.file.fullpath: %{additional.event_ProcessName}	Value taken directly from the raw log field additional.event__ProcessName.
additional.event__PrivilegeList	principal.user.attribute.permissions.name: %{additional.event__PrivilegeList}	Value taken directly from the raw log field additional.event__PrivilegeList.
additional.event__ProviderName	additional.fields.key: eventProviderName additional.fields.value.string_value: %{additional.eventProviderName}	Value taken directly from the raw log field additional.event__ProviderName.
additional.event__RestrictedAdminMode	additional.fields.key: eventRestrictedAdminMode additional.fields.value.string_value: %{additional.eventRestrictedAdminMode}	Value taken directly from the raw log field additional.event__RestrictedAdminMode.
additional.event__ReturnCode	additional.fields.key: eventReturnCode additional.fields.value.string_value: %{additional.eventReturnCode}	Value taken directly from the raw log field additional.event__ReturnCode.
additional.event__RpcCallClientLocality	additional.fields.key: eventRpcCallClientLocality additional.fields.value.string_value: %{additional.eventRpcCallClientLocality}	Value taken directly from the raw log field additional.event__RpcCallClientLocality.
additional.event__Service	securityresult.description: %{additional.event_Service}	Value taken directly from the raw log field additional.event__Service.
additional.event__Status	additional.fields.key: eventStatus additional.fields.value.string_value: %{additional.eventStatus}	Value taken directly from the raw log field additional.event__Status.
additional.event__SubStatus	additional.fields.key: eventSubStatus additional.fields.value.string_value: %{additional.eventSubStatus}	Value taken directly from the raw log field additional.event__SubStatus.
additional.event__SubjectDomainName	principal.administrativedomain: %{additional.event_SubjectDomainName}	Value taken directly from the raw log field additional.event__SubjectDomainName.
additional.event__SubjectLogonId	additional.fields.key: eventSubjectLogonId additional.fields.value.string_value: %{additional.eventSubjectLogonId}	Value taken directly from the raw log field additional.event__SubjectLogonId.
additional.event__SubjectUserName	principal.user.user_displayname: %{additional.event_SubjectUserName}	Value taken directly from the raw log field additional.event__SubjectUserName.
additional.event__SubjectUserSid	principal.user.windowssid: %{additional.event_SubjectUserSid}	Value taken directly from the raw log field additional.event__SubjectUserSid.
additional.event__TaskContentNew	additional.fields.key: eventTaskContentNew additional.fields.value.string_value: %{additional.eventTaskContentNew}	Value taken directly from the raw log field additional.event__TaskContentNew.
additional.event__TaskName	additional.fields.key: eventTaskName additional.fields.value.string_value: %{additional.eventTaskName}	Value taken directly from the raw log field additional.event__TaskName.
additional.event__TargetDomainName	target.administrativedomain: %{additional.event_TargetDomainName}	Value taken directly from the raw log field additional.event__TargetDomainName.
additional.event__TargetLinkedLogonId	additional.fields.key: eventTargetLinkedLogonId additional.fields.value.string_value: %{additional.eventTargetLinkedLogonId}	Value taken directly from the raw log field additional.event__TargetLinkedLogonId.
additional.event__TargetLogonId	additional.fields.key: eventTargetLogonId additional.fields.value.string_value: %{additional.eventTargetLogonId}	Value taken directly from the raw log field additional.event__TargetLogonId.
additional.event__TargetOutboundDomainName	additional.fields.key: eventTargetOutboundDomainName additional.fields.value.string_value: %{additional.eventTargetOutboundDomainName}	Value taken directly from the raw log field additional.event__TargetOutboundDomainName.
additional.event__TargetOutboundUserName	additional.fields.key: eventTargetOutboundUserName additional.fields.value.string_value: %{additional.eventTargetOutboundUserName}	Value taken directly from the raw log field additional.event__TargetOutboundUserName.
additional.event__TargetSid	target.user.windowssid: %{additional.event_TargetSid}	Value taken directly from the raw log field additional.event__TargetSid.
additional.event__TargetUserName	target.user.userid: %{additional.event__TargetUserName}	Value taken directly from the raw log field additional.event__TargetUserName.
additional.event__TargetUserSid	target.user.windowssid: %{additional.event_TargetUserSid}	Value taken directly from the raw log field additional.event__TargetUserSid.
additional.event__TokenElevationType	additional.fields.key: eventTokenElevationType additional.fields.value.string_value: %{additional.eventTokenElevationType}	Value taken directly from the raw log field additional.event__TokenElevationType.
additional.event__TransmittedServices	additional.fields.key: eventTransmittedServices additional.fields.value.string_value: %{additional.eventTransmittedServices}	Value taken directly from the raw log field additional.event__TransmittedServices.
additional.event__VirtualAccount	additional.fields.key: eventVirtualAccount additional.fields.value.string_value: %{additional.eventVirtualAccount}	Value taken directly from the raw log field additional.event__VirtualAccount.
additional.event__WorkstationName	target.hostname: %{additional.event__WorkstationName}	Value taken directly from the raw log field additional.event__WorkstationName.
additional.event_id	security_result.rule_name: EventID: %{additional.event_id}	Value taken directly from the raw log field additional.event_id and converted to a string.
additional.query	network.dns.questions.name: %{additional.query}	Value taken directly from the raw log field additional.query.
additional.response	network.dns.answers.name: %{additional.response}	Value taken directly from the raw log field additional.response.
metadata.description	metadata.description: %{metadata.description}	Value taken directly from the raw log field metadata.description.
metadata.eventTimestamp	metadata.event_timestamp.seconds: Extracted from %{metadata.eventTimestamp} metadata.event_timestamp.nanos: Extracted from %{metadata.eventTimestamp}	Seconds and nanoseconds are extracted from the raw log field metadata.eventTimestamp using date parsing.
metadata.eventType	metadata.product_event_type: %{metadata.eventType} metadata.event_type: %{metadata.eventType}	Value taken directly from the raw log field metadata.eventType.
metadata.logType	metadata.product_event_type: %{metadata.logType} metadata.event_type: %{metadata.logType}	Value taken directly from the raw log field metadata.logType.
network.applicationProtocol	network.application_protocol: %{network.applicationProtocol}	Value taken directly from the raw log field network.applicationProtocol.
network.direction	network.direction: %{network.direction}	Value taken directly from the raw log field network.direction.
network.ipProtocol	network.ip_protocol: %{network.ipProtocol}	Value taken directly from the raw log field network.ipProtocol.
principal.assetId	principal.asset_id: TANIUM:%{principal.assetId}	Value taken directly from the raw log field principal.assetId and prefixed with "TANIUM:".
principal.hostname	principal.hostname: %{principal.hostname}	Value taken directly from the raw log field principal.hostname.
principal.process.companySpecificParentProcessId	principal.process.product_specific_process_id: TANIUM:%{principal.assetId}:%{principal.process.companySpecificParentProcessId}	Value taken directly from the raw log field principal.process.companySpecificParentProcessId and formatted as "TANIUM:%{principal.assetId}:%{principal.process.companySpecificParentProcessId}".
principal.process.companySpecificProcessId	principal.process.product_specific_process_id: TANIUM:%{principal.assetId}:%{principal.process.companySpecificProcessId}	Value taken directly from the raw log field principal.process.companySpecificProcessId and formatted as "TANIUM:%{principal.assetId}:%{principal.process.companySpecificProcessId}".
principal.process.commandLine	target.process.command_line: %{principal.process.commandLine}	Value taken directly from the raw log field principal.process.commandLine with double quotes removed and hyphens replaced with ampersands.
principal.process.file.fullPath	target.process.file.full_path: %{principal.process.file.fullPath}	Value taken directly from the raw log field principal.process.file.fullPath.
principal.process.file.md5	target.process.file.md5: %{principal.process.file.md5}	Value taken directly from the raw log field principal.process.file.md5 and converted to lowercase.
principal.process.parentPid	principal.process.pid: %{principal.process.parentPid}	Value taken directly from the raw log field principal.process.parentPid only for PROCESS_LAUNCH events.
principal.process.pid	target.process.pid: %{principal.process.pid}	Value taken directly from the raw log field principal.process.pid only for PROCESS_LAUNCH events.
principal.process.productSpecificProcessId	principal.process.product_specific_process_id: TANIUM:%{principal.assetId}:%{principal.process.productSpecificProcessId}	Value taken directly from the raw log field principal.process.productSpecificProcessId and formatted as "TANIUM:%{principal.assetId}:%{principal.process.productSpecificProcessId}".
principal.user.groupid	principal.user.group_identifiers: %{principal.user.groupid}	Value taken directly from the raw log field principal.user.groupid.
principal.user.userid	principal.user.userid: %{principal.user.userid}	Value taken directly from the raw log field principal.user.userid.
src.ip	principal.ip: %{src.ip}	Value taken directly from the raw log field src.ip.
src.port	principal.port: %{src.port}	Value taken directly from the raw log field src.port and converted to an integer.
target.file.fullPath	target.file.full_path: %{target.file.fullPath}	Value taken directly from the raw log field target.file.fullPath.
target.file.md5	target.file.md5: %{target.file.md5}	Value taken directly from the raw log field target.file.md5.
target.port	target.port: %{target.port}	Value taken directly from the raw log field target.port and converted to an integer.
target.registry.registryKey	target.registry.registry_key: %{target.registry.registryKey}	Value taken directly from the raw log field target.registry.registryKey.
target.registry.registryValue	target.registry.registry_value_data: %{target.registry.registryValue}	Value taken directly from the raw log field target.registry.registryValue.
target.user.userDisplayName	target.user.user_display_name: %{target.user.userDisplayName}	Value taken directly from the raw log field target.user.userDisplayName.
target.user.windowsSid	target.user.windows_sid: %{target.user.windowsSid}	Value taken directly from the raw log field target.user.windowsSid.
user-agent	network.http.user_agent: %{user-agent}	Value taken directly from the raw log field user-agent with double quotes removed.
N/A	extensions.auth.auth_mechanism: LOCAL/NETWORK/BATCH/SERVICE/UNLOCK/NETWORK_CLEAR_TEXT/NEW_CREDENTIALS/REMOTE_INTERACTIVE/CACHED_INTERACTIVE/MECHANISM_UNSPECIFIED	Determined by the parser code based on the value of additional.event__LogonType.
N/A	extensions.auth.type: MACHINE	Added by the parser code for USER_LOGIN and USER_LOGOUT events.
N/A	metadata.event_type: PROCESS_LAUNCH/NETWORK_CONNECTION/FILE_OPEN/FILE_DELETION/REGISTRY_MODIFICATION/USER_LOGIN/STATUS_UPDATE/USER_LOGOUT/PROCESS_MODULE_LOAD/PROCESS_TERMINATION/USER_CHANGE_PERMISSIONS/SCHEDULED_TASK_MODIFICATION/SCHEDULED_TASK_DISABLE/SCHEDULED_TASK_ENABLE/SCHEDULED_TASK_DELETION/SCHEDULED_TASK_CREATION/PROCESS_UNCATEGORIZED	Determined by the parser code based on the value of metadata.logType and additional.event_id.
N/A	metadata.log_type: TANIUM_TH	Hardcoded value added by the parser code.
N/A	metadata.product_name: Stream	Hardcoded value added by the parser code.
N/A	metadata.vendor_name: Tanium	Hardcoded value added by the parser code.
N/A	principal.hostname: %{principal_hostname}	Value taken from either principal.hostname or additional.event__FQDN.
N/A	principal.ip: %{srcIp}	Extracted from the raw log message using grok if src.ip is present.
N/A	securityresult.about.resource.name: %{additional.event_AuthenticationPackageName}	Value taken directly from the raw log field additional.event__AuthenticationPackageName for specific additional.event_id values.
N/A	security_result.category: AUTH_VIOLATION	Added by the parser code for specific additional.event_id values.
N/A	security_result.rule_name: EventID: %{additional.event_id}	Value taken directly from the raw log field additional.event_id and converted to a string.
N/A	target.hostname: %{query_host}	Extracted from the raw log field additional.query if it contains a hostname.
N/A	target.ip: %{dstIp}	Extracted from the raw log message using grok if src.ip is present.
N/A	target.ip: %{query_ip}	Extracted from the raw log field additional.query if it contains an IP address.
N/A	target.process.command_line: %{principal_process_commandLine}	Value taken from principal.process.commandLine if it is not empty.
N/A	target.process.file.full_path: %{principal_process_file_fullPath}	Value taken from principal.process.file.fullPath if it is not empty.
N/A	target.process.file.md5: %{principal_process_file_md5}	Value taken from principal.process.file.md5 if it is not empty.
N/A	target.process.product_specific_process_id: TANIUM:%{principal.assetId}:%{principal.process.productSpecificProcessId}	Value taken from principal.process.productSpecificProcessId and formatted as "TANIUM:%{principal.assetId}:%{principal.process.productSpecificProcessId}".
N/A	target.resource.resource_type: TASK	Added by the parser code for specific additional.event_id values.
N/A	timestamp.seconds: Extracted from %{metadata.eventTimestamp} timestamp.nanos: Extracted from %{metadata.eventTimestamp}	Seconds and nanoseconds are extracted from the raw log field metadata.eventTimestamp using date parsing.

Need more help? Get answers from Community members and Google SecOps professionals.