Unattended project recommender

The unattended project recommender analyzes usage activity on projects in your organization and provides recommendations that help you discover, reclaim or remove unattended projects.

Overview

Cloud projects can go abandoned, unused, or unattended for a number of reasons (for example, a one-time prototype that's temporary and no longer needed, one-time pre-sales demos, a project cancellation, a project owner switching jobs, and so on). This leads to unnecessary spending, security exposure, and operational overhead. Identifying and remediating unattended projects can be challenging as it requires analysis of networking, API activity, cloud services, and other signals.

To address these challenges, the unattended project recommender analyzes the usage activity across all projects in your organization and provides the following:

  • Usage insights for every project (networking, API, project owner, and service activity)
  • Recommendations to turn down projects having low usage activity.
  • Recommendations to assign a new owner to projects that have high usage activity but no active owner.

How it works

The unattended project recommender analyzes project usage over the last 30 days. The following table lists the recommendations and insights that the unattended project recommender can generate based on whether the usage activity is low or high .

Project usage/type Insight subtype Recommendation Recommendation subtype
Projects with:
  • Low usage for the last 30 days
PROJECT_ACTIVITY Review or delete the project. CLEANUP_PROJECT
Projects with:
  • High usage for the last 30 days
  • User owners assigned but none of whom are active on the project.
PROJECT_ACTIVITY Reclaim the project by assigning a new owner. RECLAIM_PROJECT
All other non-Apps Script projects. PROJECT_ACTIVITY - -

Ranking the usage activity of a project

A project can be ranked to have a low or high usage activity, based on which recommendations are provided. The ranking of a project's usage activity is computed based on the content included in the project's utilization insight.

For projects that are identified to have low usage activity, the following conditions apply:

Usage classification Condition
Organizations that have more than 50 projects
  • A project is ranked based on the level of usage activity across all content included in the project's utilization insight.
  • A project is classified as unattended if it falls within 10% of usage activity in comparison to all other projects in that organization.
Organizations that have less than 50 projects
  • Daily average of API calls consumed by a project (consumedApiDailyCount) is less than 100.
  • Cloud logging is the only service used by a project.
  • All other project utilization insights have a value equal to zero.

Contents of a project utilization insight

The contents of a project's utilization insight are field values that are used to rank the usage activity of the project and generate CLEANUP_PROJECT and/or RECLAIM_PROJECT recommendations.

The following table lists the various fields that the recommender surfaces as part of the PROJECT_ACTIVITY insight:

Field Description
Level of usage
usagePercentile Percentile of the usage level of this project compared with other projects within the same organization.

If the percentile is -1, this means that the value is not computed. The value is not computed when organizations have less than 50 projects.

API activity
activeServiceAccountDailyCount Daily average of service accounts with authentication activity under this project.
apiClientDailyCount Daily average of distinct clients of API calls produced by this project.
consumedApiDailyCount Daily average of API calls consumed by this project.
datastoreApiDailyCount Daily average of Datastore API calls consumed by this project
Networking activity
vpcEgressDailyBytes Daily average of VPC egress bytes out of this project.
vpcIngressDailyBytes Daily average of VPC ingress bytes into this project
Billing usage
hasBillingAccount Checks if the project has a billing account at the end of the observation
serviceWithBillableUsage Names of the billable services used in the trailing 30 days.
Cloud services usage
activeAppengineInstanceDailyCount: system/instance_count Daily average of the number of active App Engine instances under this project
activeCloudsqlInstanceDailyCount Daily average of the number of active Cloud SQL instances under this project.
activeGceInstanceDailyCount Daily average of the number of active Compute Engine instances under this project.
bigqueryInflightJobDailyCount Daily average of inflight BigQuery jobs under this project.
bigqueryInflightQueryDailyCount Daily average of inflight BigQuery query count under this project.
bigqueryStorageDailyBytes Daily average of inflight BigQuery storage under this project.
bigqueryTableDailyCount Daily average of BigQuery table count under this project.
gcsObjectDailyCount Daily average of number of Cloud Storage objects under this project.
gcsRequestDailyCount Daily average of number of Cloud Storage API under this project.
gcsStorageDailyBytes Daily average of storage bytes used by Cloud Storage under this project.
User activity
numActiveUserOwners The number of active user type project owners at the end of the observation period. Active means that the user account is not disabled and there was some activity identified under this project during the observation period.
owners List of project owners and their activities
Other fields
hasActiveOauthTokens Checks if the project has any active OAuth tokens used in the last 180 days.

Recommendations to delete a project

A project is recommended for deletion when it has a low usage in the last 30 days and no OAuth tokens used in the last 180 days.

Recommendations to assign a new owner to a project

It is recommended to assign a new owner to a project when it meets all conditions as listed below:

  • All assigned owners are either inactive on the project for the last 90 days or no longer with the company.
  • The project has a high usage activity.

Pricing

This recommender falls under the Standard recommender pricing tier. For more details on pricing tiers, refer to the recommender pricing.

Before you begin

Before you can view the insights and recommendations, you must do the following:

Description Roles
View recommendations recommender.projectUtilViewer
View and update recommendations recommender.projectUtilAdmin
Opting-out of insights and recommendations in Transparency and Control Center dataprocessing.admin

These roles provide a set of permissions that enable you to access the insights and recommendations. For more information about roles, refer to understanding roles and granting IAM permissions.

For more information about opting out, refer to opting out of insights and recommendations.

Recommender ID

Here are the unattended project recommender ID and insight type:

  • google.resourcemanager.projectUtilization.Recommender
  • google.resourcemanager.projectUtilization.Insight

Required IAM permissions

To access the list of recommendations and insights, the following permissions are required:

  • recommender.resourcemanagerProjectUtilizationRecommendations.get
  • recommender.resourcemanagerProjectUtilizationRecommendations.list
  • recommender.resourcemanagerProjectUtilizationInsights.get
  • recommender.resourcemanagerProjectUtilizationInsights.list

To update recommendations and insights (for example, to dismiss a recommendation), the following permissions are required:

  • recommender.resourcemanagerProjectUtilizationRecommendations.update
  • recommender.resourcemanagerProjectUtilizationInsights.update

Viewing unattended project recommendations

This section describes how to check for unattended insights and recommendations using gcloud, and using curl to send requests to the recommender API.

gcloud

To view recommendations and insights using gcloud, follow the steps below. For more information, see Using the API - Insights and Using the API - Recommendations.

Recommendations

  1. To list unattended project recommendations for the project where you've enabled the Recommender API, run the following command:

      gcloud recommender recommendations list \
      --project=PROJECT_ID \
      --location=global \
      --recommender=google.resourcemanager.projectUtilization.Recommender
    

    Replace the following:

    • PROJECT_ID: The project ID.
  2. To list unattended project recommendations in a different project while using a project where you've enabled the Recommender API (as in the previous step), run the following command:

      gcloud recommender recommendations list \
      --billing-project=BILLING_PROJECT_ID \
      --project=PROJECT_ID \
      --location=global \
      --recommender=google.resourcemanager.projectUtilization.Recommender
    

    Replace the following:

    • BILLING_PROJECT_ID: The ID of the project that you've enabled with the Recommender API (this would be the same PROJECT_ID as given in the previous step).
    • PROJECT_ID: The ID of a different project for which you'd like to view the unattended recommendations.

Insights

  1. To list usage insights for a given project, run the following command:

      gcloud recommender insights list \
      --project=PROJECT_ID \
      --location=global \
      --insight-type=google.resourcemanager.projectUtilization.Insight
    

    Replace the following:

    • PROJECT_ID: The project ID.
  2. To list usage insights for a different project, while using a project where you've enabled the Recommender API (as in the previous step), run the following command:

      gcloud recommender insights list \
      --billing-project=BILLING_PROJECT_ID \
      --project=PROJECT_ID \
      --location=global \
      --insight-type=google.resourcemanager.projectUtilization.Insight
    

    Replace the following:

    • BILLING_PROJECT_ID: The ID of the project that you've enabled with the Recommender API (this would be the same PROJECT_ID as given in the previous step).
    • PROJECT_ID: The ID of a different project for which you'd like to view the unattended recommender insights.

API

To view recommendations and insights, you can use curl to send a request to the recommender APIs. To authorize requests from the command line, you use OAuth access tokens. An OAuth access token is a string that grants temporary access to an API.

  1. For viewing recommendations, open a command window and copy the following command:

    curl \
    -H "Authorization: Bearer $(gcloud auth print-access-token)"  \
    -H "x-goog-user-project: PROJECT_ID" \
    "https://recommender.googleapis.com/v1/projects/PROJECT_ID/locations/global/
    recommenders/google.resourcemanager.projectUtilization.Recommender/recommendations"
    

    Replace the following:

    • PROJECT_ID: The project ID.
  2. For viewing insights, open a command window and copy the following command:

    curl \
    -H "Authorization: Bearer $(gcloud auth print-access-token)" \
    -H "x-goog-user-project: PROJECT_ID" \
    "https://recommender.googleapis.com/v1/projects/PROJECT_ID/locations/global
    /insightTypes/google.resourcemanager.projectUtilization.Insight/insights"
    

    Replace the following:

    • PROJECT_ID: The project ID.

What's next