Multi-Project Cloud Monitoring made easier
Rahul Harpalani
Product Manager
John Day
Product Marketing Manager, Google Cloud
Customers need scale and flexibility from their cloud and this extends into supporting services such as monitoring and logging. Google Cloud’s Monitoring and Logging observability services are built on the same platforms used by all of Google that handle over 16 million metrics queries per second, 2.5 exabytes of logs per month, and over 14 quadrillion metric points on disk, as of 2020. However, you let us know through consistent feedback that the previous construct of Workspaces for Cloud Monitoring was not providing the flexibility needed for your larger scale projects.
Cloud Operation’s New Approach to Multi-Project Monitoring
We’re happy to announce a new model for multi-project monitoring, which replaces the concept of Workspaces. This overhaul is geared toward maximizing the flexibility you have to manage your monitoring environments by introducing Metrics Scopes. Starting today you can associate your Google Cloud projects with multiple Metrics Scopes!
Like Workspaces, Metrics Scopes will still be used to store all of the configuration content for dashboards, alerting policies, uptime checks, notification channels, and group definitions. However there is no limit to the number of Metrics Scopes to which you can associate a project. Prior to this change, a project could only be scoped with a single Workspace. Now, there are virtually unlimited possibilities for how you can set up multi-project monitoring. This unlocks a large variety of options, from more granular permissions to mission-focused configurations. At its most simple implementation though: operators/SREs can now create org-wide Metrics Scopes with monitoring configurations focused on infrastructure health. And developers can leverage Metrics Scopes built on a subset of their organization’s projects that allow them to focus on their application’s performance.How it works
- When you have a collection of projects, Metrics Scopes enable you to view each project's metrics in isolation as well as in combination with metrics stored by other projects.
- The Metrics Scope is hosted by a scoping project. This scoping project is the Cloud project that is selected in the Cloud Console project picker.
Example
- In this example, Project-SRE is the name of a scoping project to monitor your fleet. You added two developer teams’ projects: Project-Dev-1 and Project-Dev-2, to Project-SRE’s Metrics Scope. If you select Project-SRE with the Cloud Console project picker and then go to the Monitoring page, you view the metrics for all three projects:
- If you select Project-Dev-1 with the Cloud Console project picker and then go to the Monitoring page, you view the Metrics Scope for Project-Dev-1 and you can only see the metrics for that project:
What else is new?
- Metrics Scopes can now monitor up to 375 projects (up from 100).
- New projects automatically start working in Cloud Monitoring without the previous 60-second Workspace creation process.
- If you want to monitor more than one project simply add it to your Metrics Scope:
Navigation
- Mentioned earlier, the Project Picker in the Cloud Console can be used to navigate between Metrics Scopes in Cloud Monitoring:
- This is now consistent with many other services across Google Cloud. Specifically, you can see how the project picker stays consistent when navigating from Cloud Monitoring to Cloud Logging:
- Additionally, to make your navigation between Metrics Scopes easy we’ve added the new Metrics Scope Tab and Panel in the UI:
Coming Soon
- The Metrics Scope API is coming within the next quarter! This API will enable you to programmatically manage your monitoring configurations and Metrics Scopes.
Current Workspaces users
If you are already using Workspaces in Cloud Monitoring you may have noticed that they converted to Metrics Scopes weeks ago. There is no additional action required and you can start taking advantage of the additional features of Metrics Scopes today.
Get Started
Companies that are digitally native or in the process of digital transformation have placed an increased operational role on developers and this often creates overlapping sets of responsibilities with Operations and SRE teams. Now multiple developer teams can focus on optimizing the performance of their applications while operators can take a fleet-wide view when maintaining and improving the performance of all of the infrastructure under their purview.
For information on configuring a Metrics Scope to include metrics for multiple projects, see Viewing metrics for multiple projects.
