Collect Ansible AWX logs

Supported in:

Google SecOps SIEM

Note: This feature is covered by Pre-GA Offerings Terms of the Google Security Operations Service Specific Terms. Pre-GA features might have limited support, and changes to pre-GA features might not be compatible with other pre-GA versions. For more information, see the Google Security Operations Technical Support Service guidelines and the Google Security Operations Service Specific Terms.

Overview

This parser extracts JSON-formatted log data from Ansible AWX, converting it into the Unified Data Model (UDM). It handles various log formats, extracts relevant fields like timestamps, IP addresses, host information, and event details, and maps them to corresponding UDM fields, also handling specific edge cases and enriching the data with labels and metadata.

Before you begin

• Ensure that you have a Google SecOps instance.
• Ensure that you have privileged access to Ansible AWX.

Configure a feed in Google SecOps to ingest Ansible AWX logs

1. Go to SIEM Settings > Feeds.
2. Click Add new.
3. In the Feed name field, enter a name for the feed (for example, Ansible AWX Logs).
4. Select Webhook as the Source type.
5. Select Ansible AWX as the Log type.
6. Click Next.
7. Optional: Specify values for the following input parameters:
   • Split delimiter: the delimiter that is used to separate log lines, such as \n.
   • Asset namespace: the asset namespace.
   • Ingestion labels: the label applied to the events from this feed.
8. Click Next.
9. Review the feed configuration in the Finalize screen, and then click Submit.
10. Click Generate Secret Key to generate a secret key to authenticate this feed.
11. Copy and store the secret key. You cannot view this secret key again. If needed, you can regenerate a new secret key, but this action makes the previous secret key obsolete.
12. On the Details tab, copy the feed endpoint URL from the Endpoint Information field. You need to specify this endpoint URL in your client application.
13. Click Done.

Create an API key for the webhook feed

1. Go to Google Cloud console > Credentials.

   Go to Credentials

2. Click Create credentials, and then select API key.

3. Restrict the API key access to the Google Security Operations API.

Specify the endpoint URL

1. In your client application, specify the HTTPS endpoint URL provided in the webhook feed.

2. Enable authentication by specifying the API key and secret key as part of the custom header in the following format:

   X-goog-api-key = API_KEY
   X-Webhook-Access-Key = SECRET
   

   Recommendation: Specify the API key as a header instead of specifying it in the URL. If your webhook client doesn't support custom headers, you can specify the API key and secret key using query parameters in the following format:

   ENDPOINT_URL?key=API_KEY&secret=SECRET
   

Replace the following:

• ENDPOINT_URL: the feed endpoint URL.
• API_KEY: the API key to authenticate to Google Security Operations.
• SECRET: the secret key that you generated to authenticate the feed.

Configure Webhook in Ansible AWX

1. Sign in to your AWX instance.
2. Go to Notifications section, located under Administration.
3. Click Add.

4. Configure Webhook Settings:

   • Name: Provide a descriptive name for the webhook (for example, Google SecOps Webhook).
   • Type: Select Webhook from the list.
   • Target URL: Enter the Google SecOps API endpoint URL.
   • HTTP Method: Select POST.

   Optional: HTTP Headers: Add any necessary headers for authentication or content type. For example, you might need to include an Authorization header with a bearer token. Consult the Google SecOps documentation for specific header requirements.

5. Click Save to create the webhook.

6. Go to to Projects, located under Resources.

7. Select the project(s) you want to associate the webhook with.

8. Within the selected project, go to the Notifications tab.

9. Select which triggers to turn ON for the Webhook notification created earlier Start | Success | Failure.

The notifications will now apply to all templates associated with the project. You can disable them manually by navigating to the Notifications field in each template.

UDM Mapping Table

Log Field	UDM Mapping	Logic
agent.ephemeral_id	observer.labels.value	The value of agent.ephemeral_id from the raw log.
agent.hostname	observer.hostname	The value of agent.hostname from the raw log.
agent.id	observer.asset_id	Concatenation of "filebeat:" and the value of agent.id from the raw log.
agent.name	observer.user.userid	The value of agent.name from the raw log.
agent.type	observer.application	The value of agent.type from the raw log.
agent.version	observer.platform_version	The value of agent.version from the raw log.
cloud.availability_zone	principal.resource.attribute.labels.value	The value of cloud.availability_zone from the raw log.
cloud.instance.id	principal.resource.product_object_id	The value of cloud.instance.id from the raw log.
cloud.instance.name	principal.resource.name	The value of cloud.instance.name from the raw log.
cloud.machine.type	principal.resource.attribute.labels.value	The value of cloud.machine.type from the raw log.
cloud.provider	principal.resource.attribute.labels.value	The value of cloud.provider from the raw log.
event1	metadata.description	The value of event1 from the raw log. If event1 is not present, the value of message is used.
event1_data.host	principal.hostname	The value of event1_data.host from the raw log. Used as a fallback if host_name and host are not present.
event1_data.remote_addr	principal.ip	The IP address extracted from event1_data.remote_addr using a grok pattern.
event1_data.task	security_result.detection_fields.value	The value of event1_data.task from the raw log.
event1_data.task_path	principal.process.file.full_path	The value of event1_data.task_path from the raw log.
event1_data.task_uuid	security_result.detection_fields.value	The value of event1_data.task_uuid from the raw log.
event1_data.uuid	metadata.product_log_id	The value of event1_data.uuid from the raw log.
event1_display	security_result.description	The value of event1_display from the raw log.
host	principal.hostname	The value of host from the raw log. Used as a fallback if host_name is not present. If it's a number, it's converted to a string.
host.architecture	target.asset.hardware.cpu_platform	The value of host.architecture from the raw log.
host.fqdn	target.administrative_domain	The value of host.fqdn from the raw log.
host.hostname	target.hostname	The value of host.hostname from the raw log.
host.id	target.asset.asset_id	"Host Id: " concatenated with the value of host.id from the raw log.
host.ip	target.asset.ip	The values of the host.ip array from the raw log.
host.mac	target.mac	The values of the host.mac array from the raw log.
host.os.codename	target.asset.attribute.labels.value	The value of host.os.codename from the raw log.
host.os.kernel	target.platform_patch_level	The value of host.os.kernel from the raw log.
host.os.name	target.asset.attribute.labels.value	The value of host.os.name from the raw log.
host.os.platform	target.platform	If the value of host.os.platform is "debian", the UDM field is set to "LINUX".
host.os.version	target.platform_version	The value of host.os.version from the raw log.
host_name	principal.hostname	The value of host_name from the raw log.
input.type	network.ip_protocol	If the value of input.type is "tcp", the UDM field is set to "TCP".
level	security_result.severity	Based on the value of level: "DEBUG", "INFO", "AUDIT" map to "INFORMATIONAL"; "ERROR" maps to "ERROR"; "WARNING" maps to "MEDIUM".
level	security_result.severity_details	The value of level from the raw log.
log.source.address	principal.ip	The IP address extracted from log.source.address using a grok pattern.
log.source.address	principal.port	The port extracted from log.source.address using a grok pattern.
logger_name	intermediary.application	The value of logger_name from the raw log.
message	metadata.description	The value of message from the raw log. Used as a fallback if event1 is not present.
metadata.event_type	metadata.event_type	Determined by parser logic based on the presence of certain fields. "NETWORK_CONNECTION" if log.source.address and host.ip are present. "STATUS_UPDATE" if principal_hostname or event1_data.remote_addr are present. "GENERIC_EVENT" otherwise.
metadata.log_type	metadata.log_type	Hardcoded to "ANSIBLE_AWX".
metadata.product_name	metadata.product_name	Hardcoded to "ANSIBLE_AWX".
metadata.vendor_name	metadata.vendor_name	Hardcoded to "ANSIBLE_AWX".
parent_uuid	security_result.detection_fields.value	The value of parent_uuid from the raw log.
principal.resource.resource_type	principal.resource.resource_type	Hardcoded to "VIRTUAL_MACHINE" if cloud.instance.name is present.
observer.labels.key	observer.labels.key	Hardcoded to "ephemeral_id".
principal.resource.attribute.labels.key	principal.resource.attribute.labels.key	Hardcoded to "machine_type", "provider", or "availability_zone" depending on the field being mapped.
security_result.detection_fields.key	security_result.detection_fields.key	Hardcoded to "parent_uuid", "task", or "task_uuid" depending on the field being mapped.
target.asset.attribute.labels.key	target.asset.attribute.labels.key	Hardcoded to "codename" or "os_name" depending on the field being mapped.
timestamp	metadata.event_timestamp	The value of timestamp from the raw log, parsed and converted to a timestamp.
timestamp	timestamp	The value of timestamp from the raw log, parsed and converted to a timestamp.

Changes

2022-11-09

• Newly created parser.