AlloyDB Omni on Kubernetes supports various metrics for monitoring the health and performance of the database. These metrics are exposed in a format suitable for scraping by Prometheus.
Labels
AlloyDB Omni on Kubernetes exposes the following types of labels.
Resource labels
AlloyDB Omni on Kubernetes exposes the following resource labels that uniquely identify the database container that the metrics belong to. These resource labels match the names of the Kubernetes resource that owns the database container:
| Label key | Label value |
|---|---|
| dbnamespace | Namespace of the dbcluster CR |
| dbcluster | Name of the dbcluster CR |
| dbinstance | Name of the dbinstance CR. Only the dbinstance of ReadPool type is supported. If the database container does not belong to a ReadPool dbinstance, this value is n/a |
| dbnode | Name of the instance CR. Every instance CR has a one-to-one mapping to a database container. |
System metadata labels
System metadata labels will change dynamically when the roles of
the DB container changes. For example, When your dbcluster is promoted from
secondary to primary, dbcluster_type will change from Secondary to Primary.
| Label key | Label value |
|---|---|
| dbcluster_type | Disaster recovery (DR) role of the dbcluster CR. Can be Primary or Secondary |
| dbinstance_type | Type of the dbinstance CR. If the container belongs to a ReadPool dbinstance, this value is ReadPool, otherwise this value is n/a |
| dbnode_type | HA role of the dbnode, can be Primary or Standby |
Metric Labels
The specific labels of each metric are listed in the following tables. For
example, database means the name of a Postgres database hosted inside the AlloyDB Omni database container.
Metrics
AlloyDB Omni on Kubernetes exposes the following metrics.
The metrics list mentions only metrics labels. All metrics start with alloydb_omni.
To learn more about metric types, see Metric types.
Database container-level Metrics
These metrics are collected per AlloyDB Omni database container. All these metrics have resource and system metadata labels.
| Name | Description | label | Unit | Type |
|---|---|---|---|---|
| alloydb_omni_database_postgresql_backends | The number of active and idle connections to the AlloyDB Omni instance. | gauge | ||
| alloydb_omni_database_postgresql_max_connections | The current value of the Postgres max_connections runtime parameter. | gauge | ||
| alloydb_omni_database_postgresql_up | Whether the Postgres main process is running, 1 if running, 0 if down. | gauge | ||
| alloydb_omni_database_postgresql_uptime_second | Time elapsed since Postgres main process starts. | second | gauge | |
| alloydb_omni_database_postgresql_vacuum_oldest_transaction_age | The current age of the oldest uncommitted transaction that is blocking vacuum operation. It's measured in the number of transactions that started after the oldest transaction. | type: one of [running, prepared, replication_slot, replica] | gauge | |
| alloydb_omni_database_postgresql_vacuum_transaction_id_utilization_percentage | The ratio of transaction ID space consumed. 1 means 100%. | gauge | ||
| alloydb_omni_instance_postgresql_backends_by_state | Current number of connections. | state: State of the connections, one of [idle, active, idle_in_transaction, idle_in_transaction_aborted, disabled, fastpath_function_call]. | gauge | |
| alloydb_omni_instance_postgresql_backends_for_top_applications | Current number of connections per application. | application_name: Name of the application | gauge | |
| alloydb_omni_instance_postgresql_blks_hit_count_total | Total number of times Postgres found the requested block in the buffer cache. | counter | ||
| alloydb_omni_instance_postgresql_blks_read_count_total | Total number of blocks read by Postgres that were not in the Postgres buffer cache. | counter | ||
| alloydb_omni_instance_postgresql_committed_transactions_count_total | Total number of transactions committed. | counter | ||
| alloydb_omni_instance_postgresql_deadlock_count_total | Number of deadlocks detected. | counter | ||
| alloydb_omni_instance_postgresql_new_connections_count_total | Total number of new connections. | counter | ||
| alloydb_omni_instance_postgresql_rolledback_transactions_count_total | Total number of transactions rolled back. | counter | ||
| alloydb_omni_instance_postgresql_temp_bytes_written_count_total | Total amount of data written to temporary files by queries. | byte | counter | |
| alloydb_omni_instance_postgresql_temp_files_written_count_total | Total number of temporary files used for writing data while performing internal algorithms. | counter | ||
| alloydb_omni_instance_postgresql_tuples_deleted_count_total | Total number of rows deleted. | counter | ||
| alloydb_omni_instance_postgresql_tuples_fetched_count_total | Total number of rows fetched. | counter | ||
| alloydb_omni_instance_postgresql_tuples_inserted_count_total | Total number of rows inserted. | counter | ||
| alloydb_omni_instance_postgresql_tuples_returned_count_total | Total number of rows returned. | counter | ||
| alloydb_omni_instance_postgresql_tuples_updated_count_total | Total number of rows updated. | counter | ||
| alloydb_omni_instance_postgresql_wait_count_total | Total wait count for a wait event. |
|
counter | |
| alloydb_omni_instance_postgresql_wait_time_second_total | Total time elapsed on a wait event. |
|
second | counter |
| alloydb_omni_instance_postgresql_replication_flush_lag_ms | Time elapsed between flushing recent WAL locally and receiving notification that the replica server has written and flushed it (but not yet applied it). |
|
ms | gauge |
| alloydb_omni_instance_postgresql_replication_replay_lag_ms | Time elapsed between flushing recent WAL locally and receiving notification that the replica server has written, flushed and applied it. |
|
ms | gauge |
| alloydb_omni_instance_postgresql_replication_write_lag_ms | Time elapsed between flushing recent WAL locally and receiving notification that the replica server has written it (but not yet flushed it or applied it). |
|
ms | gauge |
| alloydb_omni_memory_available_byte | Estimate of the amount of memory available for allocation. | byte | gauge | |
| alloydb_omni_node_cpu_mcpu | Number of mCPUs allocated. 1000 mCPU = 1 CPU | mCPU | gauge | |
| alloydb_omni_node_cpu_usage_second_total | Total CPU seconds used. | second | counter | |
| alloydb_omni_node_network_received_bytes_count_total | Network received bytes count on the AlloyDB Omni pod. | byte | counter | |
| alloydb_omni_node_network_sent_bytes_count_total | Network sent bytes count on the AlloyDB Omni pod. | byte | counter | |
| alloydb_omni_node_storage_limit_per_disk_byte | Storage limit in byte. | disk: Name of the disk | byte | gauge |
| alloydb_omni_node_storage_read_bytes_count_total | Number of bytes read from disk. | byte | counter | |
| alloydb_omni_node_storage_read_ops_count_total | Number of disk read IO operations. | counter | ||
| alloydb_omni_node_storage_usage_per_disk_byte | Storage used in bytes per disk. | disk: Name of the disk | byte | gauge |
| alloydb_omni_node_storage_write_bytes_count_total | Number of bytes written to disk. | byte | counter | |
| alloydb_omni_node_storage_write_ops_count_total | Number of disk write IO operations. | counter |
Database-level metrics
These metrics are collected on a per AlloyDB Omni database container per Postgres database level. You can create multiple Postgres databases in one database container. All these metrics have resource, system metadata, and "database" labels. The database label is the name of the Postgres database that the metric belongs to.
| Name | Description | label | Unit | Type |
|---|---|---|---|---|
| alloydb_omni_database_postgresql_backends_for_top_databases | The current number of connections per database. | gauge | ||
| alloydb_omni_database_postgresql_blks_hit_for_top_databases_count_total | Total number of times Postgres found the requested block in the buffer cache per database. | counter | ||
| alloydb_omni_database_postgresql_blks_read_for_top_databases_count_total | Total number of blocks read by Postgres that were not in the Postgres buffer cache per database. | counter | ||
| alloydb_omni_database_postgresql_committed_transactions_for_top_databases_count_total | Total number of transactions committed per database. | counter | ||
| alloydb_omni_database_postgresql_deadlock_for_top_databases_count_total | The number of deadlocks per database. | counter | ||
| alloydb_omni_database_postgresql_insights_aggregate_execution_time_us_total | Total execution time across all queries. |
|
us | counter |
| alloydb_omni_database_postgresql_insights_aggregate_io_time_us_total | Total time spent doing IO for across all queries. |
|
us | counter |
| alloydb_omni_database_postgresql_new_connections_for_top_databases_count_total | The number of new connections per database. | counter | ||
| alloydb_omni_database_postgresql_rolledback_transactions_for_top_databases_count_total | Total number of transactions rolled back per database. | counter | ||
| alloydb_omni_database_postgresql_size_byte | Size of the database. | byte | gauge | |
| alloydb_omni_database_postgresql_statements_executed_count_total | Total count of statements executed per database. | operation_type: Name of the operation, one of [SELECT, UPDATE, INSERT, DELETE, MERGE, UTILITY, NOTHING, UNKNOWN]. | counter | |
| alloydb_omni_database_postgresql_temp_bytes_written_for_top_databases_count_total | Total amount of data written to temporary files by queries per database. | byte | counter | |
| alloydb_omni_database_postgresql_temp_files_written_for_top_databases_count_total | Total number of temporary files used for writing data while performing internal algorithms per database. | counter | ||
| alloydb_omni_database_postgresql_tuples | Number of rows in the database. | state: one of [live, dead] | gauge | |
| alloydb_omni_database_postgresql_tuples_deleted_for_top_databases_count_total | The total number of rows deleted per database. | counter | ||
| alloydb_omni_database_postgresql_tuples_fetched_for_top_databases_count_total | The total number of rows fetched per database. | counter | ||
| alloydb_omni_database_postgresql_tuples_inserted_for_top_databases_count_total | The total number of rows inserted per database. | counter | ||
| alloydb_omni_database_postgresql_tuples_returned_for_top_databases_count_total | The total number of rows returned per database. | counter | ||
| alloydb_omni_database_postgresql_tuples_updated_for_top_databases_count_total | The total number of rows updated per database. | counter |
Metrics collection metrics
These metrics indicate the status of each metric collection cycle. They have the resource labels mentioned in Labels.
| Name | Description | Unit | Type |
|---|---|---|---|
| alloydb_omni_monitor_collect_ms | Number of milliseconds spent to collect metrics. | ms | gauge |
| alloydb_omni_monitor_error_count | Number of errors encountered while trying to collect metrics this cycle. | gauge | |
| alloydb_omni_monitor_metric_count | Number of metrics collected successfully this cycle. | gauge |
Prometheus Metric Handler Metrics
These metrics are automatically generated by Prometheus for each collection cycle.
| Name | Description | Cause | Type |
|---|---|---|---|
| promhttp_metric_handler_errors_total | Total number of internal errors encountered by the promhttp metric handler. | Cause of the error | counter |
What's next
- To learn how to use metrics for monitoring, see Monitor AlloyDB Omni.