Optimize your cloud by exporting Active Assist recommendations to a BigQuery dataset

Active Assist provides insights and recommendations to help Google Cloud customers proactively optimize their cloud environments for cost, security, performance, and sustainability. If you’re like most customers, you’ve likely encountered these insights and recommendations in the console — either in the Recommendations Hub or embedded on a resource page, like IAM or the VM list pages. (Note that for the rest of this post, we’ll use the word “recommendations” to mean “insights and recommendations.”)

We heard from customers who love recommendations that it should be easier to discover and work with these valuable suggestions across an entire organization, so last year we launched the Recommendations BigQuery export feature to Preview. This allows you to automatically export the recommendations to a BigQuery dataset, which you can then investigate with tools like DataStudio or Looker, and integrate with your company’s existing monitoring solutions and workflows. 

Many of you have told us how powerful this feature is, so we’ve been working hard on improvements to make it even more useful. We’re happy to announce that BQ Export 1.0 is now in GA, and that BQ Export 2.0 is in Preview, with new support for Billing Account-level recommendations and cost optimization recommendations that include discount information. This post will cover what’s new with Active Assist BQ Export, as well as a couple other new features of Active Assist.

New features to Active Assist BigQuery Export

Before we dive into the details, if you’re new to Recommendations BQ Export check out our handy getting started guide. It covers the permissions you’ll need and how to set up the export, including instructions for using a service account. It also includes some sample queries and instructions for interacting with the data using Google Sheets. You may also want to check out our prior blog post showing how to optimize your cloud spend with BigQuery and Looker.   

Now, on to what’s new!

1. Non-project scoped recommendations
You can now see non-project scoped recommendations, such as billing account, organization, and folder-level recommendations in your export to BigQuery. This ensures that in your export, you are able to see the full portfolio of recommendations that are available to you. 

2. Custom contract Pricing 
If you have any applicable custom contract pricing for your company, your cost-related recommendations now take those into account (based on your historical costs) in the cost saving calculation when you export them to BigQuery. This will ensure that you have the most accurate cost savings data we have for you and your organization to help you make the best judgment and prioritization when choosing to adopt recommendations. However, you must have the correct permissions first in order to see the custom contract pricing. If the user executing the export to BigQuery does not have the correct permissions, you may continue to see list pricing in your estimated savings. Also, note that the console UI and the Recommender API already have this support.

3. Export now available globally
Customers outside of the US can now set up an export of recommendations to a BigQuery dataset. 

Improvements to Active Assist discoverability and usability

1. Global Recommender Viewer role: You can now add the global Recommender Viewer role, which gives you view access to all insights and recommendations available to you, simplifying permission management for new recommenders. Newly launched recommendations will also automatically be added as they become generally available.  

2. Dismiss recommendations via Recommender API: You can now directly dismiss recommendations via our API, allowing you to focus on the recommendations you care about and work more efficiently. 

3. Shareable links: Another feature that quietly launched recently is the availability of shareable URLs that link to recommendation details in the console. You can access these links in the UI from the upper right of the details panel of any recommendation.

However, these links become even more powerful when combined with Recommendations BigQuery Export. The URLs all have a standard format. This means that within the BigQuery export tables you can easily calculate a new column containing these links using an expression like:

This example is for project level recommendations (where cloud_entity_type = PROJECT_NUMBER). 

For cloud_entity_type = FOLDER, use:

For cloud_entity_type = ORGANIZATION, use:

These links can then be embedded in your reports and dashboards, or used by other BigQuery clients. 

As a reminder, if you’re interested in setting up reports or dashboards using Recommendations BigQuery Export, take a look at this previous blog post for some great ideas, or you can reference our getting started guide. If you have any feedback, please feel free to reach out to active-assist-feedback@google.com.