Collect Zscaler Webproxy logs

Supported in:

This document describes how you can export Zscaler Webproxy logs by setting up a Google Security Operations feed and how log fields map to Google SecOps Unified Data Model (UDM) fields.

For more information, see Data ingestion to Google SecOps overview.

A typical deployment consists of Zscaler Webproxy and the Google SecOps Webhook feed configured to send logs to Google SecOps. Each customer deployment can differ and might be more complex.

The deployment contains the following components:

  • Zscaler Webproxy: The platform from which you collect logs.

  • Google SecOps feed: The Google SecOps feed that fetches logs from Zscaler Webproxy and writes logs to Google SecOps.

  • Google SecOps: Retains and analyzes the logs.

An ingestion label identifies the parser which normalizes raw log data to structured UDM format. The information in this document applies to the parser with the ZSCALER_WEBPROXY ingestion label.

Before you begin

  • Ensure that you have access to Zscaler Internet Access console. For more information, see Secure Internet and SaaS Access ZIA Help.
  • Ensure that you are using Zscaler Webproxy 2024 or later.
  • Ensure that all systems in the deployment architecture are configured with the UTC time zone.
  • Ensure that you have the API key which is needed to complete feed setup in Google SecOps. For more information, see Setting up API keys.

Set up an ingestion feed in Google SecOps to ingest Zscaler Webproxy logs

  1. Go to SIEM Settings > Feeds.
  2. Click Add new.
  3. In the Feed name field, enter a name for the feed (for example, Zscaler Webproxy Logs).
  4. Select Webhook as the Source Type.
  5. Select Zscaler as the Log Type.
  6. Click Next.
  7. Optional: Enter values for the following input parameters:
    1. Split delimiter: the delimiter that is used to separate the logs lines. Leave blank if a delimiter is not used.
    2. Asset namespace: the asset namespace.
    3. Ingestion labels: the label to be applied to the events from this feed.
  8. Click Next.
  9. Review your new feed configuration, and then click Submit.
  10. Click Generate Secret Key to generate a secret key to authenticate this feed.

Set up Zscaler Webproxy

  1. In the Zscaler Internet Access console, click Administration > Nanolog Streaming Service > Cloud NSS Feeds and then click Add Cloud NSS Feed.
  2. The Add Cloud NSS Feed window appears. In the Add Cloud NSS Feed window, enter the details.
  3. Enter a name for the feed in the Feed Name field.
  4. Select NSS for Web in NSS Type.
  5. Select the status from the Status list to activate or deactivate the NSS feed.
  6. Keep the value in the SIEM Rate drop-down as Unlimited. To suppress the output stream due to licensing or other constraints, change the value.
  7. Select Other in the SIEM Type list.
  8. Select Disabled in the OAuth 2.0 Authentication list.
  9. Enter a size limit for an individual HTTP request payload to the SIEM's best practice in Max Batch Size. For example, 512 KB.
  10. Enter the HTTPS URL of the Chronicle API endpoint in the API URL in the following format:

      https://<CHRONICLE_REGION>-chronicle.googleapis.com/v1alpha/projects/<GOOGLE_PROJECT_NUMBER>/locations/<LOCATION>/instances/<CUSTOMER_ID>/feeds/<FEED_ID>:importPushLogs
    
    • CHRONICLE_REGION: Region where your Chronicle instance is hosted. For example, US.
    • GOOGLE_PROJECT_NUMBER: BYOP project number. Obtain this from C4.
    • LOCATION: Chronicle region. For example, US.
    • CUSTOMER_ID: Chronicle customer ID. Obtain from C4.
    • FEED_ID: Feed ID shown on Feed UI on the new webhook created
    • Sample API URL:
    https://us-chronicle.googleapis.com/v1alpha/projects/12345678910/locations/US/instances/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/feeds/yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy:importPushLogs
    
  11. Click Add HTTP Header to add more HTTP headers with keys and values.

    For example, Header 1: Key1: X-goog-api-key and Value1: API Key generated on Google Cloud BYOP's API Credentials.

  12. Select Web Logs in the Log Types list.

  13. Select JSON in the Feed Output Type list.

  14. Set Feed Escape Character to , \ ".

  15. To add a new field to the Feed Output Format, select Custom in the Feed Output Type list.

  16. Copy-paste the Feed Output Format and add new fields. Ensure the key names match the actual field names.

  17. Following is the default Feed Output Format:

      \{ "sourcetype" : "zscalernss-web", "event" : \{"datetime":"%d{yy}-%02d{mth}-%02d{dd} %02d{hh}:%02d{mm}:%02d{ss}","reason":"%s{reason}","event_id":"%d{recordid}","protocol":"%s{proto}","action":"%s{action}","transactionsize":"%d{totalsize}","responsesize":"%d{respsize}","requestsize":"%d{reqsize}","urlcategory":"%s{urlcat}","serverip":"%s{sip}","requestmethod":"%s{reqmethod}","refererURL":"%s{ereferer}","useragent":"%s{eua}","product":"NSS","location":"%s{elocation}","ClientIP":"%s{cip}","status":"%s{respcode}","user":"%s{elogin}","url":"%s{eurl}","vendor":"Zscaler","hostname":"%s{ehost}","clientpublicIP":"%s{cintip}","threatcategory":"%s{malwarecat}","threatname":"%s{threatname}","filetype":"%s{filetype}","appname":"%s{appname}","pagerisk":"%d{riskscore}","threatseverity":"%s{threatseverity}","department":"%s{edepartment}","urlsupercategory":"%s{urlsupercat}","appclass":"%s{appclass}","dlpengine":"%s{dlpeng}","urlclass":"%s{urlclass}","threatclass":"%s{malwareclass}","dlpdictionaries":"%s{dlpdict}","fileclass":"%s{fileclass}","bwthrottle":"%s{bwthrottle}","contenttype":"%s{contenttype}","unscannabletype":"%s{unscannabletype}","deviceowner":"%s{deviceowner}","devicehostname":"%s{devicehostname}","keyprotectiontype":"%s{keyprotectiontype}"\}\}
    
  18. Select the timezone for the Time field in the output file in the Timezone list. By default, the timezone is set to your organization's time zone.

  19. Review the configured settings.

  20. Click Save to test connectivity. If the connection is successful, a green tick accompanied by the message Test Connectivity Successful: OK (200) appears.

For more information about Google SecOps feeds, see Google SecOps feeds documentation. For information about requirements for each feed type, see Feed configuration by type.

If you encounter issues when you create feeds, contact Google SecOps support.

Field mapping reference

The following table lists the log fields of the ZSCALER_WEBPROXY log type and their corresponding UDM fields.

Log field UDM mapping Logic
metadata.vendor_name The metadata.vendor_name UDM field is set to Zscaler.
metadata.event_type If the ClientIP log field value is not empty and the serverip log field value is not empty and the proto log field value contain one of the following values, then the metadata.event_type UDM field is set to NETWORK_HTTP.
  • HTTPS
  • HTTP
Else, if the ClientIP log field value is not empty and the serverip log field value is not empty, then the metadata.event_type UDM field is set to NETWORK_CONNECTION.

Else, if the user log field value is not empty or the deviceowner log field value is not empty, then the metadata.event_type UDM field is set to USER_UNCATEGORIZED.

Else, the metadata.event_type UDM field is set to GENERIC_EVENT.
metadata.product_name The metadata.product_name UDM field is set to Web Proxy.
security_result.category If the bwthrottle log field value is equal to Yes, then the security_result.category UDM field is set to POLICY_VIOLATION.
intermediary.resource.resource_type If the rdr_rulename log field value is not empty, then the intermediary.resource.resource_type UDM field is set to GATEWAY.
network.application_protocol If the protocol log field value contain one of the following values, then the network.application_protocol UDM field is set to HTTP.
  • HTTP
  • HTTP_PROXY
Else, if the protocol log field value contain one of the following values, then the network.application_protocol UDM field is set to HTTPS.
  • HTTPS
  • SSL
  • TUNNEL_SSL
  • DNSOVERHTTPS
  • TUNNEL
Else, the network.application_protocol UDM field is set to UNKNOWN_APPLICATION_PROTOCOL.
security_result.action If the action log field value is equal to Allowed, then the security_result.action UDM field is set to ALLOW.

Else, if the action log field value is equal to Blocked, then the security_result.action UDM field is set to BLOCK.
security_result.severity If the pagerisk log field value is greater than or equal to 90 and the pagerisk log field value is less than or equal to 100, then the security_result.severity UDM field is set to CRITICAL.

If the pagerisk log field value is greater than or equal to 75 and the pagerisk log field value is less than or equal to 89, then the security_result.severity UDM field is set to HIGH.

If the pagerisk log field value is greater than or equal to 46 and the pagerisk log field value is less than or equal to 74, then the security_result.severity UDM field is set to MEDIUM.

If the pagerisk log field value is greater than or equal to 1 and the pagerisk log field value &is less than or equal to 45, then the security_result.severity UDM field is set to LOW.

If the pagerisk log field value is equal to 0, then the security_result.severity UDM field is set to NONE.
principal.asset.platform_software.platform If the deviceostype log field value matches the regular expression pattern (?i)iOS, then the principal.asset.platform_software.platform UDM field is set to IOS.

Else, if the deviceostype log field value matches the regular expression pattern (?i)Android, then the principal.asset.platform_software.platform UDM field is set to ANDROID.

Else, if the deviceostype log field value matches the regular expression pattern (?i)Windows, then the principal.asset.platform_software.platform UDM field is set to WINDOWS.

Else, if the deviceostype log field value matches the regular expression pattern (?i)MAC, then the principal.asset.platform_software.platform UDM field is set to MAC.

Else, if the deviceostype log field value matches the regular expression pattern (?i)Other, then the principal.asset.platform_software.platform UDM field is set to UNKNOWN_PLATFORM.
action security_result.action_details
alpnprotocol additional.fields[alpnprotocol]
app_risk_score target.security_result.risk_score If the app_risk_score log field value matches the regular expression pattern [0-9]+, then the app_risk_score log field is mapped to the security_result.risk_score UDM field.
appclass target.security_result.detection_fields[appclass]
appname target.application
apprulelabel target.security_result.rule_name
b64apprulelabel target.security_result.rule_name
b64dept principal.user.department
b64filename target.file.full_path
b64fwd_gw_name intermediary.resource.name
b64host target.hostnametarget.asset.hostname
b64login principal.user.email_addresses
b64mobappname additional.fields[mobappname]
b64rdr_rulename intermediary.security_result.rule_name
b64referer network.http.referral_url
b64rulelabel security_result.rule_name
b64threatname security_result.threat_name
b64ua network.http.user_agent
b64ua network.http.parsed_user_agent
b64upload_filename target.file.full_path
b64url target.url
b64urlcat security_result.category_details
b64urlfilterrulelabel security_result.rule_name
b64userlocationname principal.location.name
b64zpa_app_seg_name additional.fields[zpa_app_seg_name]
bamd5 target.file.md5
bwclassname security_result.detection_fields[bwclassname]
bwrulename security_result.rule_name
bwthrottle security_result.detection_fields[bwthrottle]
bypassed_etime security_result.detection_fields[bypassed_etime]
bypassed_traffic security_result.detection_fields[bypassed_traffic]
ClientIP principal.ip
clientpublicIP principal.nat_ip
clientsslcipher network.tls.client.supported_ciphers
clientsslsessreuse security_result.detection_fields[clientsslsessreuse]
clienttlsversion network.tls.version
cloudname principal.user.attribute.labels[cloudname]
clt_sport principal.port
cltsslfailcount security_result.detection_fields[cltsslfailcount]
cltsslfailreason security_result.detection_fields[cltsslfailreason]
company principal.user.company_name
contenttype additional.fields[contenttype]
cpubip additional.fields[cpubip]
datacenter target.location.name
datacentercity target.location.city
datacentercountry target.location.country_or_region
datetime metadata.event_timestamp
day additional.fields[day]
dd additional.fields[dd]
department principal.user.department
deviceappversion additional.fields[deviceappversion]
devicehostname principal.asset.hostname
devicemodel principal.asset.hardware.model
devicename principal.asset.asset_id
deviceosversion principal.asset.software.version
deviceowner principal.user.userid
devicetype principal.asset.category
df_hosthead security_result.detection_fields[df_hosthead]
df_hostname security_result.detection_fields[df_hostname]
dlpdicthitcount security_result.detection_fields[dlpdicthitcount]
dlpdictionaries security_result.detection_fields[dlpdictionaries]
dlpengine security_result.detection_fields[dlpengine]
dlpidentifier security_result.detection_fields[dlpidentifier]
dlpmd5 security_result.detection_fields[dlpmd5]
dlprulename security_result.rule_name
edepartment principal.user.department
eedone additional.fields[eedone]
efilename target.file.full_path
ehost target.hostnametarget.asset.hostname
elocation principal.location.name
elogin principal.user.email_addresses
emobappname additional.fields[mobappname]
ereferer network.http.referral_url
erefererhost additional.fields[refererhost]
erefererpath additional.fields[erefererpath]
erulelabel security_result.rule_name
eua network.http.user_agent
eua network.http.parsed_user_agent
eupload_filename target.file.full_path
eurl target.url
eurlfilterrulelabel security_result.rule_name
eurlpath additional.fields[eurlpath]
euserlocationname principal.location.name
event_id metadata.product_log_id
external_devid additional.fields[external_devid]
externalspr security_result.about.artifact.last_https_certificate.extension.certificate_policies
fileclass additional.fields[fileclass]
filename target.file.full_path
filesubtype additional.fields[filesubtype]
filetype target.file.mime_type
flow_type additional.fields[flow_type]
fwd_gw_ip intermediary.ip
fwd_gw_name intermediary.resource.name
fwd_type intermediary.resource.attribute.labels[fwd_type]
hh additional.fields[hh]
hostname target.hostnametarget.asset.hostname
is_sslexpiredca security_result.detection_fields[is_sslexpiredca]
is_sslselfsigned security_result.detection_fields[is_sslselfsigned]
is_ssluntrustedca security_result.detection_fields[is_ssluntrustedca]
keyprotectiontype security_result.about.artifact.last_https_certificate.extension.key_usage
location principal.location.name
mm additional.fields[mm]
mobappcat additional.fields[mobappcat]
mobappname additional.fields[mobappname]
mobdevtype additional.fields[mobdevtype]
module target.security_result.detection_fields[module]
mon additional.fields[mon]
mth additional.fields[mth]
nsssvcip about.ip
oapprulelabel security_result.detection_fields[oapprulelabel]
obwclassname security_result.detection_fields[obwclassname]
ocip security_result.detection_fields[ocip]
ocpubip additional.fields[ocpubip]
odevicehostname security_result.detection_fields[odevicehostname]
odevicename security_result.detection_fields[odevicename]
odeviceowner security_result.detection_fields[odeviceowner]
odlpdict security_result.detection_fields[odlpdict]
odlpeng security_result.detection_fields[odlpeng]
odlprulename security_result.detection_fields[odlprulename]
ofwd_gw_name security_result.detection_fields[ofwd_gw_name]
ologin additional.fields[ologin]
ordr_rulename additional.fields[ordr_rulename]
ourlcat security_result.detection_fields[ourlcat]
ourlfilterrulelabel security_result.detection_fields[ourlfilterrulelabel]
ozpa_app_seg_name additional.fields[ozpa_app_seg_name]
pagerisk security_result.risk_score
productversion metadata.product_version
rdr_rulename intermediary.security_result.rule_name
reason security_result.description If the action log field value is equal to Blocked, then the reason log field is mapped to the security_result.description UDM field.
refererhost additional.fields[refererhost]
refererURL network.http.referral_url
reqdatasize additional.fields[reqdatasize]
reqhdrsize additional.fields[reqhdrsize]
requestmethod network.http.method
requestsize network.sent_bytes
reqversion additional.fields[reqversion]
respdatasize additional.fields[respdatasize]
resphdrsize additional.fields[resphdrsize]
responsesize network.received_bytes
respversion additional.fields[respversion]
rulelabel security_result.rule_name If the action log field value is equal to Blocked, then the rulelabel log field is mapped to the security_result.rule_name UDM field.
ruletype security_result.rule_type
serverip target.ip
serversslsessreuse security_result.detection_fields[server_ssl_sess_reuse]
sha256 target.file.sha256
sourcetype additional.fields[sourcetype]
srvcertchainvalpass security_result.detection_fields[srvcertchainvalpass]
srvcertvalidationtype security_result.detection_fields[srvcertvalidationtype]
srvcertvalidityperiod security_result.detection_fields[srvcertvalidityperiod]
srvocspresult security_result.detection_fields[srvocspresult]
srvsslcipher network.tls.cipher
srvtlsversion security_result.detection_fields[srvtlsversion]
srvwildcardcert security_result.detection_fields[srvwildcardcert]
ss additional.fields[ss]
ssldecrypted security_result.detection_fields[ssldecrypted]
status network.http.response_code
threatcategory security_result.associations.name
threatclass security_result.associations.description
threatname security_result.threat_name
throttlereqsize security_result.detection_fields[throttlereqsize]
throttlerespsize security_result.detection_fields[throttlerespsize]
trafficredirectmethod intermediary.resource.attribute.labels[trafficredirectmethod]
transactionsize additional.fields[transactionsize]
tz additional.fields[tz]
ua_token additional.fields[ua_token]
uaclass additional.fields[uaclass]
unscannabletype security_result.detection_fields[unscannabletype]
upload_doctypename additional.fields[upload_doctypename]
upload_fileclass additional.fields[upload_fileclass]
upload_filename target.file.full_path If the filename log field value is equal to None and the upload_filename log field value is not equal to None, then the upload_filename log field is mapped to the target.file.full_path UDM field.
upload_filesubtype additional.fields[upload_filesubtype]
upload_filetype target.file.mime_type If the filetype log field value is equal to None and the upload_filetype log field value is not equal to None, then the upload_filetype log field is mapped to the target.file.mime_type UDM field.
url target.url
urlcategory security_result.category_details
urlcatmethod security_result.detection_fields[urlcatmethod]
urlclass security_result.detection_fields[urlclass]
urlfilterrulelabel security_result.rule_name
urlsupercategory security_result.category_details
user principal.user.email_addresses
useragent network.http.user_agent
useragent network.http.parsed_user_agent
userlocationname principal.location.name If the userlocationname log field value is not equal to None, then the userlocationname log field is mapped to the principal.location.name UDM field.
yyyy additional.fields[yyyy]
zpa_app_seg_name additional.fields[zpa_app_seg_name]
ztunnelversion additional.fields[ztunnelversion]

Need more help? Get answers from Community members and Google SecOps professionals.