Managing insights

Beta — service account insights

This feature is covered by the Pre-GA Offerings Terms of the Google Cloud Platform Terms of Service. Pre-GA features may have limited support, and changes to pre-GA features may not be compatible with other pre-GA versions. For more information, see the launch stage descriptions.

In addition to providing recommendations, Recommender uses machine learning (ML) to provide detailed insights. Insights are findings that highlight notable patterns in resource usage. For example, you can collect additional information about permission usage in your project, or identify unused service accounts. Some insights also link to recommendations, because the insights provide evidence for the recommendations.

Identity and Access Management (IAM) provides the following insight types:

• Policy insights: ML-based findings about permission usage within your project.
• Service account insights: Findings about service accounts in your project, such as which service accounts have not been used in the past 90 days.

Before you begin

• If you want to use policy insights, be familiar with the IAM recommender.
• Optional: Read about Recommender insights.

Required permissions

The required permissions for using insights vary depending on what you want to do.

Permissions to view insights

To view insights, you need a role that includes the following permissions, where insight-type is either iamPolicyInsights or iamServiceAccountInsights:

• recommender.insight-type.get
• recommender.insight-type.list

To gain these permissions while following the principle of least privilege, ask your administrator to grant you one of the following roles:

• IAM Recommender Viewer (roles/recommender.iamViewer)
• IAM Security Reviewer (roles/iam.securityReviewer)

Alternatively, your administrator can grant you a different role with the required permissions, such as a custom role or a more permissive predefined role.

Permissions to modify insights

To modify insights, you need a role that includes the following permissions, where insight-type is either iamPolicyInsights or iamServiceAccountInsights:

• recommender.insight-type.get
• recommender.insight-type.list
• recommender.insight-type.update

To gain these permissions while following the principle of least privilege, ask your administrator to grant you the IAM Recommender Admin role (roles/recommender.iamAdmin).

Alternatively, your administrator can grant you a different role with the required permissions, such as a custom role or a more permissive predefined role.

Listing insights

To list all insights of a specific type for your project, use one of the following methods:

gcloud

Use the gcloud recommender insights list command to view all insights of a specific type for your project.

Replace insight-type-id with the insight type you want to list. Use google.iam.policy.Insight for policy insights and google.iam.serviceAccount.Insight for service account insights.

gcloud recommender insights list --insight-type=insight-type-id \
    --location=global

The output lists all of the insights of a specific type for your project. For example, listing all policy insights for your project outputs a result like the following:

INSIGHT_ID                            LOCATION  INSIGHT_TYPE               CATEGORY  INSIGHT_STATE  LAST_REFRESH_TIME
07841f74-02ce-4de8-bbe6-fc4eabb68568  global    google.iam.policy.Insight  SECURITY  ACCEPTED       2020-07-12T07:00:00Z
0d3ce433-f067-4e78-b6ae-03d7d1f6f040  global    google.iam.policy.Insight  SECURITY  ACTIVE         2020-07-13T07:00:00Z
0e2cc488-38fb-4b9b-942c-cfe06a0ab88f  global    google.iam.policy.Insight  SECURITY  ACTIVE         2020-07-13T07:00:00Z
12b557be-d48f-49cf-a0b0-b3b73a178edf  global    google.iam.policy.Insight  SECURITY  ACTIVE         2020-07-13T07:00:00Z
279ef748-408f-44db-9a4a-1ff8865b9839  global    google.iam.policy.Insight  SECURITY  ACTIVE         2020-07-13T07:00:00Z
29a4553d-9ffb-4508-9f13-77f40fc4e8b6  global    google.iam.policy.Insight  SECURITY  ACTIVE         2020-07-13T07:00:00Z
2a00a91a-3e37-4dca-81f7-fb607d18053f  global    google.iam.policy.Insight  SECURITY  ACTIVE         2020-07-13T07:00:00Z
2baea818-df89-4ab3-8a48-0e752459d816  global    google.iam.policy.Insight  SECURITY  ACTIVE         2020-07-13T07:00:00Z
4a59da9d-cde8-46cc-9c68-6980487175fb  global    google.iam.policy.Insight  SECURITY  ACTIVE         2020-07-13T07:00:00Z
78fee8d9-c25c-4070-9f1b-ae5e4a4a164b  global    google.iam.policy.Insight  SECURITY  ACTIVE         2020-07-13T07:00:00Z

REST

The Recommender API's insights.list method lists all insights of a specific type for your project.

Before using any of the request data below, make the following replacements:

• project-id: Your Google Cloud project ID.
• insight-type-id: The type of the insight that you want to list. For policy insights, use google.iam.policy.Insight. For service account insights, use google.iam.serviceAccount.Insight.

HTTP method and URL:

GET https://recommender.googleapis.com/v1/projects/project-id/locations/global/insightTypes/insight-type-id/insights

To send your request, expand one of these options:

curl (Linux, macOS, or Cloud Shell)

Note: Ensure you have set the GOOGLE_APPLICATION_CREDENTIALS environment variable to your service account private key file path.

Execute the following command:

curl -X GET \
-H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
https://recommender.googleapis.com/v1/projects/project-id/locations/global/insightTypes/insight-type-id/insights

PowerShell (Windows)

Note: Ensure you have set the GOOGLE_APPLICATION_CREDENTIALS environment variable to your service account private key file path.

Execute the following command:

$cred = gcloud auth application-default print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
  -Method GET `
  -Headers $headers `
  -Uri "https://recommender.googleapis.com/v1/projects/project-id/locations/global/insightTypes/insight-type-id/insights" | Select-Object -Expand Content

The response lists all of the insights of a specific type in your project. For example, it could list all of the policy insights in your project:

{
  "insights": [
    {
      "name": "projects/123456789012/locations/global/insightTypes/google.iam.policy.Insight/insights/07841f74-02ce-4de8-bbe6-fc4eabb68568",
      "description": "0 permission checks were authorized by this policy binding tuple.",
      "content": {
        "role": "roles/viewer",
        "member": "serviceAccount:[e-mail]",
        "condition": {
          "expression": "",
          "title": "",
          "description": "",
          "location": ""
        },
        "exercisedPermissions": [],
        "inferredPermissions": []
      },
      "lastRefreshTime": "2020-07-12T07:00:00Z",
      "observationPeriod": "7776000s",
      "stateInfo": {
        "state": "ACTIVE"
      },
      "category": "SECURITY",
      "associatedRecommendations": [
        {
          "recommendation": "projects/123456789012/locations/global/recommenders/google.iam.policy.Recommender/recommendations/b1932220-867d-43d1-bd74-fb95876ab656"
        }
      ],
      "targetResources": [
        "//cloudresourcemanager.googleapis.com/projects/1234567890"
      ],
      "insightSubtype": "PERMISSIONS_USAGE",
      "etag": "\"b153ab487e4ae100\""
    },
    {
      "name": "projects/123456789012/locations/global/insightTypes/google.iam.policy.Insight/insights/f4292f55-105b-4744-9dc3-fcacf59685bb",
      "description": "4 permission checks were authorized by this policy binding tuple.",
      "content": {
        "role": "roles/owner",
        "member": "serviceAccount:[e-mail]",
        "condition": {
          "expression": "",
          "title": "",
          "description": "",
          "location": ""
        },
        "exercisedPermissions": [
          {
            "permission": "iam.roles.create"
          },
          {
            "permission": "iam.roles.delete"
          },
          {
            "permission": "iam.roles.list"
          },
          {
            "permission": "iam.roles.update"
          }
        ],
        "inferredPermissions": []
      },
      "lastRefreshTime": "2020-07-12T07:00:00Z",
      "observationPeriod": "7776000s",
      "stateInfo": {
        "state": "ACTIVE"
      },
      "category": "SECURITY",
      "associatedRecommendations": [
        {
          "recommendation": "projects/123456789012/locations/global/recommenders/google.iam.policy.Recommender/recommendations/6ab16c1d-edce-45e5-8d82-570fdd49892a"
        }
      ],
      "targetResources": [
        "//cloudresourcemanager.googleapis.com/projects/1234567890"
      ],
      "insightSubtype": "PERMISSIONS_USAGE",
      "etag": "\"49bb705553338fc3\""
    }
  ]
}

To learn more about the components of an insight, see Reviewing insights on this page.

Getting a single insight

To get more information about a single insight, including the insight's description, status, and any recommendations it's associated with, use one of the following methods:

gcloud

Use the gcloud recommender insights describe command with your insight ID to view information about a single IAM insight.

Before using this sample, make the following replacements:

• insight-id: The ID of the insight that you want to view. To find the ID, list the insights in your project.

• insight-type-id: The insight type you want to view. Use google.iam.policy.Insight for policy insights and google.iam.serviceAccount.Insight for service account insights.

gcloud recommender insights describe insight-id \
    --insight-type=insight-type-id --location=global

The output shows the insight in detail. For example, the following insight indicates that [e-mail] has used zero permissions from the Viewer role (roles/viewer) in the past 90 days:

associatedRecommendations:
- recommendation: projects/123456789012/locations/global/recommenders/google.iam.policy.Recommender/recommendations/0573b702-96a5-4622-a916-c762e7b0731f
category: SECURITY
content:
  condition:
    description: ''
    expression: ''
    location: ''
    title: ''
  exercisedPermissions: []
  inferredPermissions: []
  member: serviceAccount:[e-mail]
  role: roles/viewer
description: 0 permission checks were authorized by this policy binding tuple.
etag: '"d3cdec23cc712bd0"'
insightSubtype: PERMISSIONS_USAGE
lastRefreshTime: '2020-07-11T07:00:00Z'
name: projects/123456789012/locations/global/insightTypes/google.iam.policy.Insight/insights/0d3ce433-f067-4e78-b6ae-03d7d1f6f040
observationPeriod: 7776000s
stateInfo:
  state: ACTIVE
targetResources:
- //cloudresourcemanager.googleapis.com/projects/1234567890

To learn more about the components of an insight, see Reviewing insights on this page.

REST

The Recommender API's insights.get method gets a single insight.

Before using any of the request data below, make the following replacements:

• project-id: Your Google Cloud project ID.
• insight-type-id: The type of the insight that you want to view. For policy insights, use google.iam.policy.Insight. For service account insights, use google.iam.serviceAccount.Insight.
• insight-id: The ID of the insight that you want to view. If you don't know the insight ID, you can find it by listing the insights in your project. The ID of an insight is everything after insights/ in the name field for the insight.

HTTP method and URL:

GET https://recommender.googleapis.com/v1/projects/project-id/locations/global/insightTypes/insight-type-id/insights/insight-id

To send your request, expand one of these options:

curl (Linux, macOS, or Cloud Shell)

Note: Ensure you have set the GOOGLE_APPLICATION_CREDENTIALS environment variable to your service account private key file path.

Execute the following command:

curl -X GET \
-H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
https://recommender.googleapis.com/v1/projects/project-id/locations/global/insightTypes/insight-type-id/insights/insight-id

PowerShell (Windows)

Note: Ensure you have set the GOOGLE_APPLICATION_CREDENTIALS environment variable to your service account private key file path.

Execute the following command:

$cred = gcloud auth application-default print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
  -Method GET `
  -Headers $headers `
  -Uri "https://recommender.googleapis.com/v1/projects/project-id/locations/global/insightTypes/insight-type-id/insights/insight-id" | Select-Object -Expand Content

The response contains the insight. For example:

{
  "name": "projects/123456789012/locations/global/insightTypes/google.iam.policy.Insight/insights/07841f74-02ce-4de8-bbe6-fc4eabb68568",
  "description": "0 permission checks were authorized by this policy binding tuple.",
  "content": {
    "role": "roles/viewer",
    "member": "serviceAccount:[e-mail]",
    "condition": {
      "expression": "",
      "title": "",
      "description": "",
      "location": ""
    },
    "exercisedPermissions": [],
    "inferredPermissions": []
  },
  "lastRefreshTime": "2020-07-12T07:00:00Z",
  "observationPeriod": "7776000s",
  "stateInfo": {
    "state": "ACTIVE"
  },
  "category": "SECURITY",
  "associatedRecommendations": [
    {
      "recommendation": "projects/123456789012/locations/global/recommenders/google.iam.policy.Recommender/recommendations/b1932220-867d-43d1-bd74-fb95876ab656"
    }
  ],
  "targetResources": [
    "//cloudresourcemanager.googleapis.com/projects/123456789012"
  ],
  "insightSubtype": "PERMISSIONS_USAGE",
  "etag": "\"b153ab487e4ae100\""
}

To learn more about the components of an insight, see Reviewing insights on this page.

Reviewing insights

Insights are divided into insight types and insight subtypes. Insight types tell you which resource type the insight is for. Insight subtypes tell you what kind of information the insight contains.

IAM offers the following insight types and subtypes:

Insight type	Insight subtype
Policy insights	PERMISSION_USAGE: ML-based findings about permission usage within your project.
Service account insights	SERVICE_ACCOUNT_USAGE: Findings about which service accounts in your project have not been used in the past 90 days.

An insight's contents are determined by its subtype. The insight subtypes listed in the preceding table have the following components:

• associatedRecommendations: The identifiers for any recommendations associated with the insight. If there are no recommendations associated with the insight, this field is empty.
• category: The category for PERMISSION_USAGE and SERVICE_ACCOUNT_USAGE insights is always SECURITY.

• content: The information in the content field is determined by the insight subtype:

  Insight subtype	Content
  PERMISSION_USAGE	Reports a member's permission usage for a specific role. This field contains the following components: member: The member whose permission usage was analyzed. role: The role for which the permission usage was analyzed. condition: Any conditions attached to binding that grants the member the role. If there are no conditions, this field contains an empty condition. exercisedPermissions: The permissions in the role that the member used during the observation period. inferredPermissions: The permissions in the role that the IAM recommender has determined, through ML, that the member is likely to need based on their exercised permissions.
  SERVICE_ACCOUNT_USAGE	Reports the last time the service account was authenticated. This field contains the following components: serviceAccountId: The unique numeric ID of the service account. email: The email address of the service account. lastAuthenticatedTime: The most recent time that the service account was authenticated. If the service account does not have any recorded authentications, this field is not included.

• description: A human-readable summary of the insight.

• etag: A unique fingerprint that identifies the current state of an insight. Each time the insight changes, a new etag value is assigned.

  To change the state of an insight, you must provide the etag of the existing insight. Using etags helps ensure that any operations are performed only if the insight has not changed since you last retrieved it.

• insightSubtype: The insight subtype, which is either PERMISSION_USAGE or SERVICE_ACCOUNT_USAGE.

• lastRefreshTime: The date when the insight was last refreshed, which indicates the freshness of the data used to generate the insight.

• name: The name of the insight, in the following format:

  projects/project-id/locations/global/insightTypes/insight-type-id/insights/insight-id

  The placeholders have the following values:

  • project-id: The ID of the project where the insight was generated.
  • insight-type-id: The type of the insight. Policy insights have the type google.iam.policy.Insight. Service account insights have the type google.iam.serviceAccount.Insight.
  • insight-id: A unique ID for the insight.

• observationPeriod: The time period leading up to the insight. The source data used to generate the insight ends at lastRefreshTime and begins at lastRefreshTime minus observationPeriod.

• stateInfo: Insights go through multiple state transitions after they are proposed:

  • ACTIVE: The insight has been generated, but either no actions have been taken, or an action was taken without updating the insight's state. Active insights are updated when the underlying data changes.
  • ACCEPTED: Some action has been taken based on the insight. Insights become accepted when an associated recommendation was marked CLAIMED, SUCCEEDED, or FAILED, or the insight was accepted directly. When an insight is in the ACCEPTED state, the content of the insight cannot change. Accepted insights are retained for 90 days after they are accepted.

• targetResources: The full resource name of the Google Cloud project that the insight is for. For example, //cloudresourcemanager.googleapis.com/projects/1234567890

Marking an insight as ACCEPTED

If you take action based on an active insight, you can mark that insight as ACCEPTED. The ACCEPTED state tells the Recommender API that you have taken action based on this insight, which helps refine your recommendations.

Accepted insights are retained for 90 days after they are marked as ACCEPTED.

gcloud

Use the gcloud recommender insights mark-accepted command with your insight ID to mark an insight as ACCEPTED.

Before using the sample command, replace the following values:

• insight-id: The ID of the insight that you want to mark as ACCEPTED. To find the ID, list the insights in your project.

• insight-type-id: The type of the insight that you want to mark as ACCEPTED. For policy insights, use google.iam.policy.Insight. For service account insights, use google.iam.serviceAccount.Insight.

• etag: An identifier for a version of the insight. To get the etag, do the following:

  1. Get the insight using the gcloud recommender insights describe command.
  2. Find and copy the etag value from the output, including the enclosing quotes. For example, "d3cdec23cc712bd0".

gcloud recommender insights mark-accepted insight-id \
    --insight-type=insight-type-id --location=global \
    --etag=etag

The output shows the insight, now with the state of ACCEPTED:

associatedRecommendations:
- recommendation: projects/123456789012/locations/global/recommenders/google.iam.policy.Recommender/recommendations/0573b702-96a5-4622-a916-c762e7b0731f
category: SECURITY
content:
  condition:
    description: ''
    expression: ''
    location: ''
    title: ''
  exercisedPermissions: []
  inferredPermissions: []
  member: serviceAccount:[e-mail]
  role: roles/viewer
description: 0 permission checks were authorized by this policy binding tuple.
etag: '"d3cdec23cc712bd0"'
insightSubtype: PERMISSIONS_USAGE
lastRefreshTime: '2020-07-11T07:00:00Z'
name: projects/123456789012/locations/global/insightTypes/google.iam.policy.Insight/insights/0d3ce433-f067-4e78-b6ae-03d7d1f6f040
observationPeriod: 7776000s
stateInfo:
  state: ACCEPTED
targetResources:
- //cloudresourcemanager.googleapis.com/projects/1234567890

To learn more about the state info of an insight, see Reviewing insights on this page.

REST

The Recommender API's insights.markAccepted method marks an insight as ACCEPTED.

Before using any of the request data below, make the following replacements:

• project-id: Your Google Cloud project ID.
• insight-type-id: The type of the insight that you want to mark as ACCEPTED. For policy insights, use google.iam.policy.Insight. For service account insights, use google.iam.serviceAccount.Insight.
• insight-id: The ID of the insight that you want to mark as ACCEPTED. If you don't know the insight ID, you can find it by listing the insights in your project. The ID of an insight is everything after insights/ in the name field for the insight.
• etag: An identifier for a version of the insight. To get the etag, do the following:
  1. Get the insight using the insights.get method.
  2. Find and copy the etag value from the response.

HTTP method and URL:

POST https://recommender.googleapis.com/v1/projects/project-id/locations/global/insightTypes/insight-type-id/insights/insight-id:markAccepted

Request JSON body:

{
  "etag": "etag"
}

To send your request, expand one of these options:

curl (Linux, macOS, or Cloud Shell)

Note: Ensure you have set the GOOGLE_APPLICATION_CREDENTIALS environment variable to your service account private key file path.

Save the request body in a file called request.json, and execute the following command:

curl -X POST \
-H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
https://recommender.googleapis.com/v1/projects/project-id/locations/global/insightTypes/insight-type-id/insights/insight-id:markAccepted

PowerShell (Windows)

Note: Ensure you have set the GOOGLE_APPLICATION_CREDENTIALS environment variable to your service account private key file path.

Save the request body in a file called request.json, and execute the following command:

$cred = gcloud auth application-default print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
  -Method POST `
  -Headers $headers `
  -ContentType: "application/json; charset=utf-8" `
  -InFile request.json `
  -Uri "https://recommender.googleapis.com/v1/projects/project-id/locations/global/insightTypes/insight-type-id/insights/insight-id:markAccepted" | Select-Object -Expand Content

The response contains the insight, now with the state of ACCEPTED:

{
  "name": "projects/1234567890/locations/global/insightTypes/google.iam.policy.Insight/insights/07841f74-02ce-4de8-bbe6-fc4eabb68568",
  "description": "0 permission checks were authorized by this policy binding tuple.",
  "content": {
    "role": "roles/viewer",
    "member": "serviceAccount:[e-mail]",
    "condition": {
      "expression": "",
      "title": "",
      "description": "",
      "location": ""
    },
    "exercisedPermissions": [],
    "inferredPermissions": []
  },
  "lastRefreshTime": "2020-07-12T07:00:00Z",
  "observationPeriod": "7776000s",
  "stateInfo": {
    "state": "ACCEPTED"
    },
  "category": "SECURITY",
  "associatedRecommendations": [
    {
      "recommendation": "projects/1234567890/locations/global/recommenders/google.iam.policy.Recommender/recommendations/b1932220-867d-43d1-bd74-fb95876ab656"
    }
  ],
  "targetResources": [
    "//cloudresourcemanager.googleapis.com/projects/1234567890"
  ],
  "insightSubtype": "PERMISSIONS_USAGE",
  "etag": "\"b153ab487e4ae100\""
}

To learn more about the state info of an insight, see Reviewing insights on this page.

What's next

• Learn how to view and apply recommendations.
• Use the Recommendation Hub to view and manage all recommendations for your project, including IAM recommendations.
• Monitor when a service account was last used with service account usage metrics.