Database observability for developers: introducing Cloud SQL Insights

Today we are excited to announce Cloud SQL Insights, a simple, open tool that helps developers quickly understand and resolve database performance issues on Cloud SQL. We designed Insights using open standards, with world-class visualizations and with modern architectures such as microservices in mind. 

We want all database customers to make the most of their deployments, so we've made Insights available at no additional cost for the most recent week of history—the last 7 days of metrics will be displayed in the dashboard. Insights is now available for Cloud SQL for PostgreSQL. 

Why are all my developers waiting for their turn with the DBA?

Application development teams are shipping features faster than ever before. What was once an event a few times a year has shifted to a DevOps model with multiple ships a week, or even many times a day. In addition, there are more application users spread around the globe, continuously driving workloads on your applications. When performance issues arise, it’s oftentimes the database that is the root cause. 

With DevOps, more ownership of the full development lifecycle has moved to software developers. However, for database performance, developers may not have the skills or tools to solve problems themselves. If they turn to DBAs for help, they are likely to find they are waiting their turn: DBAs are highly outnumbered and have become a scarce resource in virtually every organization. Urgent performance issues in production applications can go unresolved, leaving customers with a poor experience.

Insights helps developers address database performance themselves, allowing for faster iteration on your applications and freeing your DBAs to work on more strategic tasks.

“The number of apps we manage and the frequency at which we release features has increased dramatically with the cloud,” said Greg Cain, MLB Vice President, Baseball Data. “This makes it harder for our DBA team to keep up with performance problems. Cloud SQL Insights should help us overcome this with self-service monitoring and diagnosis for our application developers and engineers who will be able to proactively prevent query performance problems.”

Database observability across the development life cycle with open standards

Insights complements your existing APM and observability tools. Insights provides database metrics and traces through the open standard OpenTelemetry and Cloud Monitoring and Cloud Trace APIs. This makes it easy to do end-to-end tracing in your existing tools and provides a full-stack view of your environments from the application through the database. 

“As organizations are increasingly adopting managed database services in the cloud, they’re asking for better observability into their application environments,” says Steve Tack, SVP of Product Management at Dynatrace. “We are pleased to combine Dynatrace’s AI-powered observability with the open standards of Cloud SQL Insights to help enterprises run applications with the highest reliability and performance.“

Application-centric database monitoring using query tags

Traditional monitoring tools provide a query-centric view of performance. This limitation can create a disconnect between performance and application code, especially with modern architectures like MVC and microservices. 

Insights provides developers with database monitoring through the lens of the application. Developers can use tags to associate queries with specific business functions, such as payment, inventory, business analytics, and shipping. Developers can then quickly evaluate the database load contributed by specific microservices or user flows, for example.

Developers simply add tags to their application code:

And use Insights for a holistic view of performance organized by business function rather than by query. Here’s a look at database load sorted by tags:

ORMs are used in most applications to simplify database query development. One of the drawbacks of ORMs is that they can generate inefficient queries that are very difficult to diagnose. Insights provides SQL Commenter, an open-source library that auto-instruments your ORMs to help identify which application code is causing problems. Once installed, SQL Commenter automatically tags queries, alleviating the need to make any changes to application code. SQL Commenter supports many popular ORMs such as Hibernate, Spring, Express, Django, Flask, and others. Learn more about SQL Commenter.

Self-service experience for query diagnostics 

With Insights, developers can seamlessly move from detection to diagnosis using a single interface. Insights helps identify query performance problems early with pre-built dashboards. Here’s an example:

Visual query plans give insights into the root cause of performance problems:

Insights is serverless. It manages the collection of telemetry for diagnostics with minimum performance impact, reducing the time you spend managing monitoring infrastructure. All performance metrics are protected by Google Cloud’s enterprise-class security, privacy, and compliance.

Get started with Insights today

To learn more about Insights, review the documentation or try out Cloud SQL for PostgreSQL.