This page describes how to use Firewall Insights, which provides access to insights, recommendations, and metrics about your firewall rules. You can view this information on the Firewall Insights page in the Google Cloud console, as well as in several other places in the console. You can also get data by using the Recommender API or the Google Cloud CLI.
For an overview of the available insights, see the Firewall Insights overview.
For a list of firewall usage metrics, see Firewall Insights metrics.
Before you begin
Before you can use Firewall Insights, you must select a project, and then complete the required setup tasks. The prerequisites for some features differ from those of others. For details, see the following table.
Task | All metrics | Shadowed rule insights | Overly permissive rule insights | Deny rules with hits |
---|---|---|---|---|
Enable the Firewall Insights API | ✔ | ✔ | ✔ | ✔ |
Enable Firewall Rules Logging | ✔ | ✔ | ✔ | |
Enable the Recommender API | ✔ | ✔ | ||
Enable this type of insight | ✔ | ✔ | ||
Configure an observation period | ✔ | ✔ |
The following sections describe each of these tasks.
Select a project
Before you complete any prerequisites or take other actions with Firewall Insights, we recommend that you create or select a Google Cloud project. Use the following steps:
In the Google Cloud console, go to the Project selector page.
Select or create a Google Cloud project.
Make sure that billing is enabled for your Cloud project.
Enable the Firewall Insights API
Before you can perform any tasks using Firewall Insights, you must enable the Firewall Insights API. Use the following steps. Alternatively, you can use the console API Library, as described in Enabling APIs.
Console
In the Google Cloud console, go to the Firewall Insights page.
Under Firewall Insights API, click Enable.
gcloud
Run the following command:
gcloud services enable firewallinsights.googleapis.com
Enable Firewall Rules Logging
If you want to view any of the following, you must enable Firewall Rules Logging:
- Metrics about firewall rules
- Insights about overly permissive rules
or
deny
rules; these insights are known collectively as log-based insights
Firewall Insights produces metrics and log-based insights only for those rules that have logging enabled. For more information, see the Firewall Rules Logging overview.
Enable the Recommender API
You must enable the Recommender API if you want to do one of the following:
- Use shadowed rule insights
- Use overly permissive rule insights
Retrieve any data by making API calls or by using the Google Cloud CLI
Console
In the console, go to the Enable access to API page.
Make sure that the correct project is selected, and then click Next.
Click Enable.
gcloud
Run the following command:
gcloud services enable recommender.googleapis.com
Enable shadowed rule or overly permissive rule insights
Firewall Insights does not generate shadowed rule or overly permissive rule insights unless you actively enable these features on the Firewall Insights page.
After you enable either feature, you might have to wait up to 48 hours to see the generated insights.
When you create or update a firewall rule, you might have to wait up to ten days to see machine learning predictions for overly permissive rule insights. In the meantime, you can view insights based on data gathered from Firewall Rules Logging.
Console
In the Google Cloud console, go to the Firewall Insights page.
Click Configuration.
Click Enablement.
As appropriate, move the slider to Enabled or Disabled for one or both of the following:
Shadowed rule insights
Overly permissive rule insights
API
You can use the Recommender API to enable or disable shadowed rule insights and/or overly permissive rule insights. You can also use the API to set the observation period for overly permissive rule insights and retrieve configuration details.
Enable insights and set the observation period
You can enable shadowed rule insights and/or overly permissive
rule insights by using the
updateConfig
method.
To use the updateConfig
method, you must set values for all of its
parameters. This means that, at the same time you enable or disable insights,
you must also configure the observation period for overly permissive insights.
To make this type of update, use the following request.
PATCH https://recommender.googleapis.com/v1beta1/projects/PROJECT_ID/locations/global/insightTypes/google.compute.firewall.Insight/config { "name": "projects/PROJECT_ID/locations/global/insightTypes/google.compute.firewall.Insight/config", "insightTypeGenerationConfig": { "params": { "observation_period": "OBSERVATION_PERIOD_OVERLY_PERMISSIVE", "enable_shadowed_rule_insights": ENABLEMENT_SHADOWED, "enable_overly_permissive_rule_insights": ENABLEMENT_OVERLY_PERMISSIVE } }, "etag": "\"ETAG\"", }
Replace the following values:
- PROJECT_ID: the ID of your project
- OBSERVATION_PERIOD_OVERLY_PERMISSIVE: the time, in seconds, of the observation period for overly permissive rules insights
- ENABLEMENT_SHADOWED: a boolean value that represents whether shadowed rule insights are enabled
- ENABLEMENT_OVERLY_PERMISSIVE: a boolean value that represents whether overly permissive rule insights are enabled
- ETAG: the IAM policy etag value; to
retrieve the etag value, use the
getConfig
method, as described in the following section
Example
PATCH https://recommender.googleapis.com/v1beta1/projects/PROJECT_ID/locations/global/insightTypes/google.compute.firewall.Insight/config { "name": "projects/PROJECT_ID/locations/global/insightTypes/google.compute.firewall.Insight/config", "insightTypeGenerationConfig": { "params": { "observation_period": "604800s", "enable_shadowed_rule_insights": true, "enable_overly_permissive_rule_insights": true } }, "etag": "\"ETAG\"", }
Retrieve configuration details
To retrieve details about how Firewall Insights is configured, use the
getConfig
method, as shown in the following example.
GET https://recommender.googleapis.com/v1beta1/projects/PROJECT_ID/locations/global/insightTypes/google.compute.firewall.Insight/config
Configure observation periods
For some insights, you can configure an observation period—the period of
time that the insight covers. You can do this for overly permissive rule and
deny
rule insights.
For example, suppose you set the observation period for deny
rule insights
to two months. In this case, when you review the list of deny
rules with
hits, Firewall Insights shows you only those rules that had hits
at some point during the past two months. Suppose you later change the
observation period to one month. In this case, a different number of rules
might be identified, because Firewall Insights would be
analyzing a smaller period of time.
When reviewing insights and configuring observation periods, be aware of the following:
When you configure the observation period for
deny
rules with hits, Firewall Insights updates the insight results immediately.When you update the observation period for overly permissive rule insights, Firewall Insights can take up to 48 hours to update existing results. In the interim, the observation period for existing results matches the previously configured observation period.
For overly permissive insights, if no firewall rules were identified by the insight, Firewall Insights does not display the observation period that was used.
Shadowed rule insights do not have an observation period because they do not evaluate historical data. Shadowed rule analysis evaluates your existing firewall rule configuration every 24 hours.
Traffic log data from the last 24 hours may not be included when generating insights.
Console
To configure an observation period:
In the Google Cloud console, go to the Firewall Insights page.
Click Configuration.
Click Observation period.
As appropriate, set the Observation period drop-down list to the appropriate time for each of the following:
Overly permissive rule insights
Deny rule insights
API
To set the observation period for deny
rules with hits, you must use the
console. However, you can use the Recommender API to set
the observation period for overly permissive rule insights. You can also use
the API to enable insights and to retrieve configuration details.
Set the observation period and enable insights
You can set the observation period for overly permissive rules insights by
using the
updateConfig
method.
To use the updateConfig
method, you must set values for all of its
parameters. This means you must also specify whether shadowed rule insights
and overly permissive rules insights are enabled or disabled.
To make this type of update, use the following request.
PATCH https://recommender.googleapis.com/v1beta1/projects/PROJECT_ID/locations/global/insightTypes/google.compute.firewall.Insight/config { "name": "projects/PROJECT_ID/locations/global/insightTypes/google.compute.firewall.Insight/config", "insightTypeGenerationConfig": { "params": { "observation_period": "OBSERVATION_PERIOD_OVERLY_PERMISSIVE", "enable_shadowed_rule_insights": ENABLEMENT_SHADOWED, "enable_overly_permissive_rule_insights": ENABLEMENT_OVERLY_PERMISSIVE } }, "etag": "\"ETAG\"", }
Replace the following values:
- PROJECT_ID: the ID of your project
- OBSERVATION_PERIOD_OVERLY_PERMISSIVE: the time, in seconds, of the observation period for overly permissive rules insights
- ENABLEMENT_SHADOWED: a boolean value that represents whether shadowed rule insights are enabled
- ENABLEMENT_OVERLY_PERMISSIVE: a boolean value that represents whether overly permissive rule insights are enabled
- ETAG: the IAM policy etag value; to
retrieve the etag value, use the
getConfig
method, as described in the following section
Example
PATCH https://recommender.googleapis.com/v1beta1/projects/PROJECT_ID/locations/global/insightTypes/google.compute.firewall.Insight/config { "name": "projects/PROJECT_ID/locations/global/insightTypes/google.compute.firewall.Insight/config", "insightTypeGenerationConfig": { "params": { "observation_period": "604800s", "enable_shadowed_rule_insights": true, "enable_overly_permissive_rule_insights": true } }, "etag": "\"ETAG\"", }
Retrieve configuration details
To retrieve details about how Firewall Insights is configured, use the
getConfig
method, as shown in the following example.
GET https://recommender.googleapis.com/v1beta1/projects/PROJECT_ID/locations/global/insightTypes/google.compute.firewall.Insight/config
Schedule a custom refresh cycle
Set up a refresh cycle to generate shadowed rule insights for your project.
You can schedule the refresh cycle to begin on a specified date, and customize the cycle frequency. The default cycle frequency is one day (24 hours).
Console
Configure a custom refresh cycle for insights:
In the Google Cloud console, go to the Firewall Insights page.
Click Configuration.
Click Enablement.
To enable shadowed rule insights, click the toggle.
In the Start on field, enter a date from which the custom refresh cycle starts.
In the Repeat every field, select the frequency for the refresh cycle starting from the cycle start date:
- day: every 24 hours
- week: every week on the days you select
- month: every month
- quarter: every quarter
The new insight generation schedule takes effect 24 hours after saving changes to the schedule.
Use the Firewall Insights landing page
The Firewall Insights landing page in the console provides cards for all insights, including the following:
- Shadowed firewall rules
- Overly permissive rules,
including each of the following:
Allow
rules with no hitsAllow
rules with unused attributesAllow
rules with overly permissive IP address or port ranges
Deny
rules with no hits
Each card includes a list of all the rules in your project that meet the insight criteria. If you want to limit the results to one VPC network only, use the filter bar at the top of the page to select a network.
The following sections describe how to view each insight.
View shadowed firewall rules
To learn about this insight, see Shadowed rules in the Firewall Insights overview.
Console
In the console, go to the Firewall Insights page.
On the card named Shadowed rules, click View full list. In response, the console displays the Shadowed rules page, which lists all the rules that were identified as shadowed. The Insight column for each rule provides a summary of why the rule was identified as a shadowed rule.
To view more details about the shadowed rule and the rules that shadow it, click the insight.
View allow
rules with no hits
To learn about this insight, see Allow rules with no hits in the Firewall Insights overview.
Console
In the console, go to the Firewall Insights page.
On the card named Allow rules with no hit, click View full list. In response, the console displays the Allow rules with no hits page, which lists all the rules that had no hits during the observation period. The Insight column for each rule shows if the firewall rule had no hits during the observation period. The Future hit prediction column shows a prediction of future usage based on firewall rules in the same organization.
For any rule in the list, do any of the following as appropriate:
- To view the Firewall rules details page for the rule, click the name of the rule.
- To view logging for the rule, click the View audit logs link.
- To see details about the prediction, click the prediction link. In response, the Insight details panel is displayed. This panel describes the rule's main attributes. It also describes the attributes of other similar rules in the project, along with their hit counts.
View allow
rules with unused attributes
To learn about this insight, see Allow rules with unused attributes in the Firewall Insights overview.
Console
In the console, go to the Firewall Insights page.
On the card named Allow rules with unused attributes, click View full list. In response, the console displays the Allow rules with unused attributes page. This page lists all rules identified that had unused attributes during the observation period. The Insight column for each rule shows the number of attributes that were unused during the observation period.
For any rule in the list, do any of the following as appropriate:
- To view the Firewall rules details page for the rule, click the name of the rule.
- To view logging for the rule, click the View audit logs link.
- To see details about the prediction, click the prediction link. In response, the Insight details panel is displayed. This panel describes the rule's main attributes. It also describes other rules in the project that have similar attributes.
View allow
rules with overly permissive IP address or port ranges
To learn about this insight, see Allow rules with overly permissive IP address or port ranges in the Firewall Insights overview.
Be aware that your project might have firewall rules that allow access from certain IP address blocks for load balancer health checks or for other Google Cloud functionality. These IP addresses might not be hit, but they should not be removed from your firewall rules. For more information about these ranges, see the Compute Engine documentation.
Console
In the console, go to the Firewall Insights page.
On the card named Allow rules with overly permissive IP address or port ranges, click View full list. In response, the console displays a list of all the rules that had overly permissive ranges during the observation period.
For any rule in the list, do any of the following as appropriate:
- To view the Firewall rules details page for any rule, click the name of the rule.
- To view logging for the rule, click the View audit logs link.
- To see suggestions about how to narrow the range, click the link in the Insight column. In response, the Insight details panel is displayed. This panel describes the rule's main attributes. It suggests more narrowly defined IP address or port ranges that you could use.
View deny
rules with hits
To learn about this insight, see Deny rules with hits in the Firewall Insights overview.
Console
In the console, go to the Firewall Insights page.
On the card named Deny rules with hits, click View full list. In response, the console displays the Deny rules with hits page, which lists all the
deny
rules that had hits during the observation period.To review the packets dropped by a firewall, click Hit count.
Mark an insight as dismissed
If any insight isn't meaningful, or if you want to hide it for any other reason, you can dismiss it. After you dismiss an insight, the console no longer displays the insight to you or other users unless you restore it.
Console
In the console, go to the Firewall Insights page.
Find the appropriate card and click View full list.
Select the rules that you want to dismiss, and then click Dismiss.
Restore a dismissed insight
If you dismissed an insight that you later think is relevant, you or another user can restore it and make it visible in the console.
Console
In the console, go to the Firewall Insights page.
Click Dismiss History. In response, the console displays the Dismissed insights page.
Select the insights that you want to restore, and then click Restore.
Export insights
If needed, you can export insights in CSV or JSON format.
You might want to export insights for any of the following reasons:
- You need to import the data into another system.
- You want to access the data while you are offline.
You intend to disable Firewall Insights, but you want to retain access to previously generated insights.
Console
In the console, go to the Firewall Insights page.
Click Save as.
Follow the prompts to choose a format for your insights and download them.
Use the Recommendation Hub
The Recommendation Hub is a feature of the Recommender product that provides usage recommendations for Google Cloud products and services.
The console for the Recommendation Hub shows the following firewall rule insights:
- Shadowed firewall rules
- Overly permissive rules,
including each of the following:
Allow
rules with no hitsAllow
rules with unused attributesAllow
rules with overly permissive IP address or port ranges
The Recommendation Hub shows these insights along with recommendations for other products, such as Identity and Access Management (IAM) and VM Rightsizing.
For general information about Recommendation Hub, see Getting started with Recommendation Hub.
View shadowed firewall rules
To learn about this insight, see Shadowed rules in the Firewall Insights overview.
Console
In the console, go to the Recommendation Hub.
On the card named Simplify firewall configuration, click View all. The Recommendation Hub displays a page that lists shadowed rules. The Insight column for each rule provides a summary of why the rule was identified as a shadowed rule.
To view more details about the shadowed rule and the rules that shadow it, click the insight.
View overly permissive rule insights
To learn about overly permissive rule insights, see the following sections in the Firewall Insights overview:
Allow
rules with no hitsAllow
rules with unused attributesAllow
rules with overly permissive IP address or port ranges
Console
In the console, go to the Recommendation Hub.
On the card named Review overly permissive firewall rules, click View all.
Use the tabs at the top of the page to toggle between the insight types.
You can click the insight for any rule listed to understand why the rule was included in the list. The insight detail shows more information about the rule's hit count and attributes.
Mark an insight as dismissed
If any insight isn't meaningful, or if you want to hide it for any other reason, you can dismiss it. After you dismiss an insight, the console no longer displays the insight to you or other users unless you restore it.
Console
In the console, go to the Recommendation Hub.
Find the appropriate card and click View all.
If you want to dismiss an insight about an overly permissive rule, use the tabs at the top of the page to navigate to the appropriate insight category.
Select the insights that you want to dismiss, and then click Dismiss.
In the confirmation dialog, click Yes, dismss.
Restore a dismissed insight
If you have dismissed an insight that you later think is relevant, you or a user can restore it and make it visible in the console by following these steps.
Console
In the console, go to the Recommendation Hub.
Find the appropriate card and click View all.
If you want to dismiss an insight about an overly permissive rule, use the tabs at the top of the page to navigate to the appropriate insight category.
At the top of the page, click History.
Click the Dismissed tab, which shows dismissed recommendations and insights for the project.
Select the insights that you want to restore.
Click Restore.
In the confirmation dialog, click Restore.
Use the Firewall details page
For more information about this page, see Listing firewall rules for a VPC network.
List insights for a project
Console
In the console, go to the Firewall page.
For each firewall rule, view the name of available insights in the Insights column.
You can click the name of an insight to view its detail. The following sections describe how to view and interpret the detail for each type of insight.
View allow
rules with no hit in the last 24 months
Console
In the console, go to the Firewall page.
In the Last hit column, review the last time that a given firewall rule was used in the last 24 months.
View the usage history chart for a rule
Console
In the console, go to the Firewall page.
Click a firewall rule name.
In the Hit count monitoring section of the page, view the resulting chart that shows the firewall hit count for a given time period. You can select tabs for different time periods above the chart.
View deny
rules with hits for an observation period
Console
In the console, go to the Firewall page.
In the Hit count column, view the number of unique connections used for a given firewall rule in the last 24 months (default).
Use the VM network interface details page
View firewall usage on the Network interface details page for a VM.
For more information about this page, see Listing firewall rules for a network interface of a VM instance.
View rules with hits in the last 24 months
Console
In the console, go to the Compute Engine VM instances page.
Choose a VM, and on the far right of the page, click its more actions menu
.
On the menu, select View network details.
In the Firewall and routes details section, click the Firewall Rules tab.
In the Hit count column, view the hit counts for
allow
anddeny
traffic in the last 24 months for all firewall rules associated with a specific network interface.
Work with insights using gcloud
commands or the API
Firewall Insights uses Recommender commands. Recommender is a Google Cloud service that provides usage recommendations for Google Cloud products and services.
List insights
gcloud
To list insights for a project, run the following command:
gcloud beta recommender insights list \ --project=PROJECT_ID \ --location=global \ --insight-type=google.compute.firewall.Insight \ --filter=EXPRESSION \ --limit=LIMIT \ --page-size=PAGE_SIZE \ --sort-by=SORT_BY \ --format=json
Replace PROJECT_ID
with the project ID that you want to
list insights for.
The location
flag always uses the location named global
. The
insight-type
flag always uses the insight type named
google.compute.firewall.Insight
. Unless you
format the output in JSON, the command output is tabular.
The following fields are optional:
EXPRESSION
: apply this Boolean filter to each resource that you want to listIf the expression evaluates as
True
, that item is listed. For more details and examples of filter expressions, run$ gcloud topic filters
or see thegcloud topic filters
documentation.LIMIT
: the maximum number of resources to list; the default number of resources listed is unlimitedPAGE_SIZE
: the maximum number of resources to list per pageThe default page size is determined by the service; otherwise, there is no paging. Paging might be applied before or after
FILTER
andLIMIT
.SORT_BY
: a list of comma-separated field key names to sort by for a resourceThe default order is ascending. To specify a descending order, prefix a field with
~
(a tilde).
API
To get all of the insights for a Google Cloud project, make a GET
request to the projects.locations.insightTypes.insights
method:
GET https://recommender.googleapis.com/v1/{parent=projects/*/locations/global/insightTypes/*}/insights
The following example shows a sample response for this command:
insights { "name": "projects/{project_number}/locations/global/insightTypes/google.compute.firewall.Insight/insights/{insight-id}" "description": "Firewall projects/{project_id}/global/firewalls/{shadowed_firewall_name} is shadowed by projects/{project_id}/global/firewalls/{shadowing_firewall_name}." "content": { "shadowingFirewalls": [ "//compute.googleapis.com/projects/{project_id}/global/firewalls/shadowing_firewall_name1}" ] }, "lastRefreshTime": "2020-04-01T19:16:43Z", "observationPeriod": "0s", "stateInfo" { "state": "ACTIVE" } "category": "SECURITY" "targetResources":[ "//compute.googleapis.com/projects/{project_id}/global/firewalls/{shadowed_firewall_name}" ], "insightSubtype": "SHADOWED_RULE" }
Describe insights
gcloud
To list details for an insight about a particular firewall rule, run the following command:
gcloud beta recommender insights describe INSIGHT_ID \ --project=PROJECT_NAME \ --location=global \ --insight-type=google.compute.firewall.Insight
Replace the following with values for your network:
INSIGHT_ID
: the ID of the insight to describePROJECT_NAME
: the name of the project that you want to list insights for
The location
flag always uses the location named global
. The
insight-type
flag always uses the insight type named
google.compute.firewall.Insight
.
API
To get details for an insight, make a GET
request to the projects.locations.insightTypes.insights
method:
GET https://recommender.googleapis.com/v1/{name=projects/*/locations/global/insightTypes/*/insights/*} { "name": projects/PROJECT_ID/locations/LOCATION/insightTypes/INSIGHT_TYPE_ID/insights/INSIGHT_ID, {
Replace the following with values for your network:
PROJECT_ID
: the project IDLOCATION
: always use the location namedglobal
INSIGHT_TYPE_ID
: always use a value ofgoogle.compute.firewall.Insight
INSIGHT_ID
: the insight ID for the insight
What's next
- To review your VPC firewall runtime usage, clean up and optimize your firewall rule configurations, and tighten up security boundaries, see Common use cases.