Security reports in the Apigee UI

You're viewing Apigee X documentation.
View Apigee Edge documentation.

In the Security Report Jobs view, you can create reports to identify security threats to your APIs. To generate the report, Apigee scans API traffic data over a specified time interval and searches for bot indicators, such as traffic patterns or malicious agents. Apigee then displays a report showing any suspicious activity. Using this information, you can then take action to block attacks against your APIs.

You can create security reports either in the Apigee UI, as described below, or using the Advanced API Security API. If you use the UI, the data for reports is restricted to the environment you choose. However, using the API, you can also create reports for environment groups.

Bot detection rules

Before seeing how to create a report, take a look a the following rules, which Advanced API Security uses to detect bots. Each rule describes a different type of suspicious traffic:

  • Flooder - High proportion of traffic from IP in a 5-minute window:
    • Minimum number of calls from IP: 100
    • Minimum percentage of total API traffic from IP: 5
  • Brute Guesser - Larger proportion of response errors during previous 24 hours:
    • Minimum number of calls from IP: 100
    • Minimum percentage of errors: 90
  • Static Content Scraper - High proportion of response payload size from IP in a 5-minute window:
    • Minimum number of calls from IP: 100
    • Minimum response payload size: 10485760
    • Minimum percentage of total API response size from IP: 5
  • OAuth Abuser- High number of OAuth sessions with small number of user agents during previous 24 hours:
    • Minimum number of calls from IP: 100
    • Number of sessions threshold: 100
    • Number of user agents threshold: 10
  • Robot Abuser - Large number of 403 rejection errors in past 24 hours:
    Minimum 403 errors per day: 500
  • Tor Exit Nodes - Tor exit nodes IP list. A Tor exit node is the last Tor node that traffic passes through in the Tor network before exiting onto the internet.

If Advanced API Security detects an API request that meets any of the above rules, it reports it as a bot and stores the IP address where the request originated. Note that an actual traffic pattern could fit more than one of the above rules.

Once Advanced API Security has identified the IP address of a bot, it reports all API traffic coming from that address as "bot traffic."

Open the Security Report Jobs view

To open the Security Report Jobs view:

  1. Open Apigee UI in a browser.
  2. Select Analyze > API Security > Report Jobs.

This displays the main Security Report Jobs window:

Security Report Jobs main window.

Each row in the view corresponds to a security report. The following report details are displayed:

  • The Report Name.
  • The report Status, which can be one of the following:
    • running: The report is currently running and cannot be viewed yet.
    • completed: The report is completed and can be viewed or downloaded.
    • expired: The report has expired and can no longer be viewed or downloaded in the UI. To view a report after the expiration date, you must download it before that date.
  • The Start Time and End Time of the report time range.

    Note: Both the Start Time and then End Time must be in the past, and can be at most one year before the present when you create the report.

  • Submitted: The date and time the request for the report was submitted.
  • Expiration Date: Date when the report expires and can no longer be viewed in the Apigee UI. The expiration date is 7 days after the time when you created the report.
  • Actions: Click the three dot menu to display the possible actions, which are:
    • View report: Opens the report in the Apigee UI.
    • Download report: Downloads the report in a zip file.

    These actions are only available if the status of the report is Completed.

Create a new security report

To create a new security report, start by clicking +Create Report Job in the Security Report Jobs view. This opens the Create Security Report Job dialog, where you can configure the options for the report as described in the next section.

Security report options

You can specify the following options for a security report:

  • Report Name: A name for the report.
  • Report Date Range: Start time and end time for the report.

    Note: The start and end times of the report must be in the past, and at most one year in the past when the report is created.

  • Metrics: Metric for the report. You can choose from the following metrics and aggregation functions—functions that compute statistics for the metrics.
    Metric Description Aggregation functions
    bot The number of distinct IP addresses for detected bots over one-minute intervals. count_distinct
    bot_traffic The number of messages from IP addresses of detected bots over one-minute intervals. sum
    message_count

    Total number of API calls processed by Apigee in one-minute intervals.

    Note: message_count cannot be used with other metrics in the same report.

    sum
    response_size Size of the response payload returned in bytes. sum, avg, min, max
  • Dimensions: Dimensions let you group metric values together based on related subsets of the data. The following dimensions are available for security reports:

    • apiproxy
    • ax_resolved_client_ip
    • developer_app
    • target_host
    • ax_dn_region
    • envgroup_hostname
    • ax_edge_execution_fault_code
    • ax_geo_city
    • ax_geo_country
    • ax_isp
    • request_uri
    • useragent
    • ax_ua_agent_family
    • ax_ua_os_family
    • ax_ua_device_category
    • client_id
    • request_verb
    • response_status_code
    • proxy_basepath
    • proxy_pathsuffix
    • ax_ua_agent_type
    • access_token

    For more information about these, see dimensions.

    bot_reason

    In addition to the dimensions listed above, there is also a dimension specific to Advanced API Security, called bot_reason, which can be any combination of the six bot detection rules described above. For a specific bot that is detected, bot_reason consists of the subset of rules that the bot's traffic pattern matched when it was detected.

    Note: The bot_reason dimension only works with the following bot-related metrics:

    • bot
    • bot_traffic
    • response_size

    To add multiple dimensions, just click +Add a Dimension for each dimension you want to add. You can also change the order in which dimensions appear in the report by clicking the up or down arrows to the right of the dimension field.

  • Filters: Filters let you restrict results to metrics with specific values. To create a filter, set the following fields:
    • Select a name for the filter.
    • Select a comparison operator.
    • Select a value.

    See Filters.

After you have selected all the report options, click Create to create the report job.

View a completed report

Once Apigee has completed the report, you can view it by clicking the three-dot menu under Actions and selecting View report.

The following sections describe examples of security reports you can create.

Example: bot IP addresses report

The following example creates a report that shows the IP addresses of detected bots. To create the report, use the following configuration:

  • Metric: bot, the number of distinct IP addresses identified as sources of bots.
  • Aggregation function: count_distinct
  • Dimension: ax_resolved_client_ip

The completed report is shown below:

Security Report bot IP address report

Note that the table at the bottom of the report lists IP addresses that Advanced API Security has identified as bots.

Example: bot traffic by bot reason report

The next example creates a report of bot traffic—the number of requests from IP addresses that have been identified as the sources of bots—by bot_reason—the set of rules that led to the bot being detected. To create the report, use the following configurarion:

  • Metric: bot traffic
  • Aggregation function: sum
  • Dimension: bot_reason

The completed report is shown below:

Security Report bot traffic by bot reason \report

Each bot reason consists of a subset of the bot detection rules described above. As you can see in the graph, The bot reason that contributed to the largest amount of bot traffic is the following set of rules:

  • Flooder
  • Brute Guesser
  • Robot Abuser

Example: bot traffic report

The next example creates a report that isn't grouped by a dimension. If you don't want to group data by a dimension, you can set Dimension to environment. Since data is always restricted to the selected environment, this results in a report that has no grouping of data.

  • Metric: bot traffic
  • Aggregation function: sum
  • Dimension: environment
Security Report of bot traffic report

The report displays the total traffic from IP addresses that have been identified as sources of bots, for each one-minute interval throughout the report time range. Note that there is no grouping

More examples of security reports

The follow table lists some examples of security of reports that you can create using different metrics and dimensions:

Report Metrics Dimensions
All Bot Traffic & Bot Count Report for per environment bot, bot_traffic environment
Bot Traffic & Bot Count Report for different bot rules bot, bot_traffic bot_reason
Bot Traffic & Bot Count Report for different Countries bot, bot_traffic ax_geo_country
Bot Traffic & Bot Count Report for different ISPs bot, bot_traffic ax_isp
Bot Detection Report (Detailed List View) bot_traffic/td> ax_resolved_client_ip, ax_isp, bot_reason, request_uri, client_id
Bot traffic per Access Token bot_traffic access_token
Bot traffic per API proxy bot_traffic apiproxy
Bot traffic per Agent Family bot_traffic ax_ua_agent_family
Bot traffic per User Agent bot_traffic useragent
Bot traffic per Agent Type bot_traffic ax_ua_agent_type
Bot traffic per Device Category bot_traffic ax_ua_device_category
Bot traffic per OS family bot_traffic ax_ua_os_family
Bot traffic per Client ID bot_traffic client_id
Bot traffic per Proxy Basepath bot_traffic proxy_basepath
Bot traffic per Proxy Path Suffix bot_traffic proxy_pathsuffix
Bot traffic per Request URI bot_traffic request_uri
Bot traffic per Request Verb bot_traffic request_verb
Bot traffic per Response Status Code bot_traffic response_status_code

Limits on security reports

Security reports have the following limits:

  • Maximum time range for bot reports is 1 year.
  • The maximum number of metrics you can use in a report is 25, and the maximum number of dimensions you can use is 25.
  • As with the asynchronous custom reports API, there is a limit of 32 MB of data for a report. If you encounter a size limit on a report, you can either:
    • Reduce the time range of the report.
    • Split the data into smaller subsets by filtering on a set of values, and then create multiple reports, one for each subset.
  • Security reports have a processing lag of around 10 minutes on average, and 15 minutes in the worst case. So creating a report in which the End Time is less than 15 mins in the past might return incorrect results.
  • The ax_resolved_client_ip dimension can't be listed in the same report with either the ax_geo_city or ax_geo_country dimension, due to privacy concerns.
  • There are a few metrics which are available only via APIs: bot_first_detected (min), bot_last_detected (max).