Collect Cloudflare WAF logs

Note: This feature is covered by Pre-GA Offerings Terms of the Google Security Operations Service Specific Terms. Pre-GA features might have limited support, and changes to pre-GA features might not be compatible with other pre-GA versions. For more information, see the Google Security Operations Technical Support Service guidelines and the Google Security Operations Service Specific Terms.

Supported in:

Google SecOps SIEM

This document describes how you can collect Cloudflare WAF logs by setting up a Google Security Operations feed.

For more information, see Data ingestion to Google Security Operations.

An ingestion label identifies the parser which normalizes raw log data to structured UDM format. The information in this document applies to the parser with the CLOUDFLARE_WAF ingestion label.

Configure Cloudflare WAF using the API

Note: Cloudflare Logpull is not enabled by default. To activate Cloudflare Logpull for your enterprise domain, contact Cloudflare support.

After activating the Cloudflare log, use the following parameters to enable the integration:

• Auth email - the Cloudflare account email address.

• Auth key - the Cloudflare API token.

• Zone ID - the Cloudflare zone ID.

To obtain the Zone ID and Auth Key, perform the following steps:

1. Sign in to the Cloudflare dashboard.
2. Click the website that needs to be monitored.
3. Click Get your API key to get Auth Key.
4. Select API Token.
5. Click Create Token.
6. Provide the following values in the Create Token page:
   1. In the Token name field, provide the token name.
   2. In the Permissions section, do the following:
      1. Select Zone, Logs, and Read in the first, second, and third lists, respectively.
      2. Click Add more. Select Zone, Analytics, and Read in the first, second and third lists, respectively.
      3. Click Add more. Select Zone, Firewall Services and Read in the first, second and third lists, respectively.
      4. In the Zone Resources section, select Include and All zones in the first and second lists, respectively.
7. Click Continue to Summary.
8. Click Create Token.
9. Copy the displayed token, which is required to configure the Google Security Operations feed.

Note: If the token is lost, you must create a new token.

Configure Cloudflare WAF using the Cloudflare dashboard

Note: Cloudflare Logpull is not enabled by default. To activate Cloudflare Logpull for your enterprise domain, contact Cloudflare support.

1. Sign in to the Cloudflare dashboard.
2. Click the website that needs to be monitored.
3. Go to Analytics & Logs > Logpush.
4. Select Create a Logpush job.
5. In Select a destination, choose Google Cloud Storage.
6. Enter or select the following destination details:
   • Bucket - Google Cloud bucket name
   • Path - bucket location within the storage container
   • Organize logs into daily subfolders (recommended)
   • For Grant Cloudflare access to upload files to your bucket, make sure your bucket has added Cloudflare IAM as a user with a Storage Object Admin role.
   • When you are done entering the destination details, select Continue.
7. To prove ownership, Cloudflare will send a file to your designated destination. To find the token, select the Open button in the Overview tab of the ownership challenge file, then paste it into the Cloudflare dashboard to verify your access to the bucket. Enter the Ownership Token and select Continue.
8. Select the dataset to push to the storage service.
9. Configure your Logpush job:
   • Enter the Job name.
   • In If logs match, select events to include or remove from your logs. Not all datasets have this option available.
   • In Send the following fields, choose to either push all logs to your storage destination or selectively choose which logs you want to push.
10. Select Submit once you are done configuring your Logpush job.

Configure a feed in Google Security Operations to ingest Cloudflare WAF logs

1. From the Google Security Operations menu, select Settings, and then click Feeds.
2. Click Add New.
3. Select Google Cloud Storage for Source Type.
4. Select Cloudflare WAF as the Log Type to create a feed for Cloudflare WAF.
5. Click Get Service Account. Google Security Operations provides a unique service account that Google Security Operations uses to ingest data.
6. Configure access for the service account to access the Cloud Storage objects. For more information, see Grant access to the Google Security Operations service account.
7. Click Next.
8. Configure the following input parameters:
   • Storage bucket URI
   • URI is a
   • Source deletion option
9. Click Next and then click Submit.

For more information about Google Security Operations feeds, see Google Security Operations feeds documentation. For information about requirements for each feed type, see Feed configuration by type.

If you encounter issues when you create feeds, contact Google Security Operations support.

Field mapping reference

This parser extracts fields from Cloudflare Web Application Firewall (WAF) JSON logs, transforming them into the Unified Data Model (UDM). It handles various log fields, including network information, HTTP details, security results, and metadata, mapping them to corresponding UDM fields for consistent representation and analysis within Google Security Operations.

UDM mapping table

Log Field	UDM Mapping	Logic
Action	security_result.action_details	The raw log "Action" field value is mapped to security_result.action_details.
Action	security_result.action	The security_result.action field is derived based on the value of the "Action" field. "allow" maps to ALLOW. "challengeSolved", "jschallengeSolved", "managedchallengenoninteractivesolved", and "managedchallengeinteractivesolved" map to ALLOW_WITH_MODIFICATION. "drop", "block", and "connectionclose" map to BLOCK. "challengefailed" and "jschallengefailed" map to FAIL. Any other non-empty value or "unknown" maps to UNKNOWN_ACTION.
ClientASN	network.asn	The raw log "ClientASN" field value is converted to a string and mapped to network.asn.
ClientASNDescription	additional.fields.key	Set to "ClientASNDescription".
ClientASNDescription	additional.fields.value.string_value	The raw log "ClientASNDescription" field value is mapped to additional.fields.value.string_value.
ClientCountry	principal.location.country_or_region	The raw log "ClientCountry" field value is mapped to principal.location.country_or_region.
ClientIP	principal.ip	The raw log "ClientIP" field value is mapped to principal.ip.
ClientRefererHost	intermediary.hostname	The raw log "ClientRefererHost" field value is mapped to intermediary.hostname.
ClientRefererPath	network.http.referral_url	The raw log "ClientRefererPath" field value is mapped to network.http.referral_url.
ClientRequestMethod	network.http.method	The raw log "ClientRequestMethod" field value is mapped to network.http.method.
ClientRequestHost	target.hostname	The raw log "ClientRequestHost" field value is mapped to target.hostname.
ClientRequestPath	target.file.full_path	The raw log "ClientRequestPath" field value is mapped to target.file.full_path if it's not empty or "/".
ClientRequestProtocol	network.application_protocol	The protocol part of the "ClientRequestProtocol" field value (e.g., "HTTP" from "HTTP/1.1") is extracted, converted to uppercase, and mapped to network.application_protocol.
ClientRequestUserAgent	network.http.user_agent	The raw log "ClientRequestUserAgent" field value is mapped to network.http.user_agent.
Datetime	metadata.event_timestamp	The raw log "Datetime" field value is parsed as an RFC 3339 timestamp and mapped to metadata.event_timestamp.
EdgeColoCode	additional.fields.key	Set to "EdgeColoCode".
EdgeColoCode	additional.fields.value.string_value	The raw log "EdgeColoCode" field value is mapped to additional.fields.value.string_value.
EdgeResponseStatus	network.http.response_code	The raw log "EdgeResponseStatus" field value is converted to an integer and mapped to network.http.response_code.
Kind	metadata.product_event_type	The raw log "Kind" field value is mapped to metadata.product_event_type.
Metadata.filter	target.resource.attribute.labels.key	Set to "Metadata filter".
Metadata.filter	target.resource.attribute.labels.value	The raw log "Metadata.filter" field value is mapped to target.resource.attribute.labels.value.
Metadata.type	target.resource.attribute.labels.key	Set to "Metadata type".
Metadata.type	target.resource.attribute.labels.value	The raw log "Metadata.type" field value is mapped to target.resource.attribute.labels.value.
RayID	metadata.product_log_id	The raw log "RayID" field value is mapped to metadata.product_log_id.
RuleID	security_result.rule_id	The raw log "RuleID" field value is mapped to security_result.rule_id.
Source	security_result.rule_name	The raw log "Source" field value is mapped to security_result.rule_name.
N/A	metadata.vendor_name	Hardcoded to "Cloudflare".
N/A	metadata.product_name	Hardcoded to "Cloudflare log Aggregator".
N/A	metadata.log_type	Hardcoded to "CLOUDFLARE_WAF".
N/A	metadata.event_type	Determined by the parser logic based on the presence of "ClientIP", "ClientRequestHost", and the value of "app_protocol". Possible values are NETWORK_HTTP, NETWORK_CONNECTION, STATUS_UPDATE, and GENERIC_EVENT.

Changes

2023-08-30

• Initialized field "ClientRequestPath".

2023-02-02

• Validated the 'security_result' value before its getting merged to event.

2022-09-16

• Mapped the field 'Action' to 'security_result.action_details'.
• Mapped 'security_result.action' to 'ALLOW_WITH_MODIFICATION' when action contains "challengeSolved", "jschallengeSolved", "managedchallengenoninteractivesolved", "managedchallengeinteractivesolved".
• Mapped 'security_result.action' to 'BLOCK' when action contains "drop", "block", "connectionclose".
• Mapped 'security_result.action' to 'FAIL' when action contains "challengefailed", "jschallengefailed".

2022-07-25

• Description- Newly created parser