Monitoring agent for SAP HANA V2.0 planning guide

With the Google Cloud monitoring agent for SAP HANA, you can use Cloud Monitoring to display SAP HANA system metrics in the Google Cloud Console.

To collect the metrics, monitoring agent for SAP HANA queries SAP HANA by using the SQL syntax and system views that are defined by SAP in the SAP HANA SQL and System Views Reference.

You can use the SAP HANA memory utilization metrics for capacity planning or to help manage memory-based SAP HANA licensing. Correlating SAP HANA metrics with Compute Engine metrics can help you identify usage and performance trends.

For even greater insight into your SAP HANA installation, you can define custom queries to capture additional metrics.

With Cloud Monitoring, you can create dashboards to visualize your SAP HANA metrics and set up alerts based on metric thresholds. For complete information about how to set up dashboards and alerts, see the Cloud Monitoring documentation.

Changes in version 2.0

Version 2.0 of the monitoring agent for SAP HANA contains the following significant changes from version 1.0:

  • The default metrics collected by the agent have changed. To review the default metrics, see Default query reference.
  • Installation has changed:
    • You install the agent using standard operating system package management commands, yum or zypper.
    • The installed file names have changed.
    • All installed files are stored in the same single directory.
  • The configuration options have changed:
    • The agent configuration and metric queries are now all defined in a single configuration file.
    • You can use Secret Manager for password protection.
    • You can use Transport Layer Security (TLS) to secure agent communications.
  • The agent runs as a systemd service for simplified operation.
  • The agent has improved error handling and stability.

Cloud Monitoring pricing for metrics

The metrics that the monitoring agent for SAP HANA collects and sends to Monitoring are classified as chargeable metrics by Monitoring. Monitoring prices chargeable metrics by ingested volume.

The sampling interval of your queries, which determines how frequently the monitoring agent for SAP HANA queries SAP HANA for metrics, affects the volume of metrics that get sent to Monitoring.

For more information about Monitoring pricing, see Pricing for Google Cloud's operations suite.

For more information about query sampling intervals, see Sample interval for queries.

Queries: default and custom

The monitoring agent for SAP HANA collects metrics from SAP HANA by using SQL queries. The agent includes a set of default queries and you can add custom queries.

Both the default queries and the custom queries retrieve metrics that are defined by the SAP HANA system views that are documented in the SAP HANA SQL and System Views Reference.

Default query type

The default queries are built-in queries that gather the most commonly used SAP HANA metrics.

The frequency of the default queries is determined by the global sample_interval property at the beginning of the configuration.yaml file. By default this value is set to 300 seconds (5 minutes). You can modify this global default. Unlike for custom queries, you cannot set a sample interval for an individual default query.

Although you can't modify the default queries, you can disable any of the default queries that you don't need in the agent configuration file, configuration.yaml.

For more information about the default queries, the metrics that they collect, and the underlying SQL statements that they use, see Default query reference.

Custom query type

You can create custom queries by defining them in the configuration.yaml file of the monitoring agent for SAP HANA.

For custom queries, you can specify a _sample_interval_ for each query that overrides the default sample interval for all queries. For more information, see Sample interval for queries.

The configuration.yaml file includes two example custom queries that are disabled by default. You can use the provided custom queries as models for your own custom queries. The example custom queries are shown below in Example custom queries.

Error handling for custom queries

The agent does not validate the SQL in custom queries before submitting the queries to SAP HANA.

If a custom query fails with either an SQL exception or if the configured columns in a query do not match the results that are returned by SAP HANA, the agent writes an error to the log. In both cases, the agent continues to run the custom query and write error messages to the log until the custom query is corrected.

For information about the required SQL syntax for custom queries, see SAP HANA SQL and System Views Reference.

Sample interval for queries

You can define a global sample interval for all queries and an individual sample interval for custom queries. A sample interval determines the frequency at which the monitoring agent queries SAP HANA for metrics.

The default sample interval for all queries is 300 seconds (5 minutes). You can adjust the default sample interval by specifying a different value on the sample_interval property at the beginning of the configuration.yaml file.

You can override the default sample interval for an individual custom query by specifying the sample_interval property in the definition of the query.

Installation architectures

You install the monitoring agent for SAP HANA on a Compute Engine VM.

You can install the agent on the same host as SAP HANA or on a different host.

If you are monitoring SAP HANA instances in a high-availability cluster, install the agent on a host that is not a node in the cluster, so that the agent is not impacted by a failover.

The agent can also monitor SAP HANA instances that are running on an on-premises hosts or on a host on another cloud platform.

If you are monitoring SAP HANA instances in multiple Google Cloud projects, you need to install at least one monitoring agent for SAP HANA in each Google Cloud project that contains an SAP HANA instance that you are monitoring.

Supported operating systems

The monitoring agent for SAP HANA supports both the general and SAP offerings of the following operating systems:

  • Red Hat Enterprise Linux (RHEL) 7 and 8
  • SUSE Linux Enterprise Server (SLES) 12 and 15

Authentication and access

The monitoring agent for SAP HANA requires an Identity and Access Management (IAM) service account for authentication with Google Cloud and for permission to access Google Cloud resources.

IAM roles that are assigned to the service account determine which Google Cloud resources the agent has permission to interact with.

Unless you use an existing or default service account, you need to create the service account in your Google Cloud project and assign one or more roles to the service account.

Required IAM roles

Whether you use a new, existing, or default service account, the service account must include roles that grant the monitoring agent for SAP HANA the permissions that it needs.

The predefined IAM Monitoring Metric Writer role (roles/monitoring.metricWriter) grants permission to write metrics to Monitoring.

If you use Secret Manager, the predefined Secret Manager Secret Accessor role (roles/secretmanager.secretAccessor) grants permission to access secrets that are stored in Secret Manager.

Specifying a service account

If the agent is running on a Compute Engine VM, the agent uses the service account of the host VM by default. You don't need to specify a service account in the configuration file.

The host VM service account is usually a service account with limited roles that you or one of your colleagues create, but it can instead be the Compute Engine default service account, which has broad permissions by default.

If you need to use a service account other than the host VM service account, or the agent is not running on a Compute Engine VM, you need to create a service account with a JSON service account key. You store the JSON key file in an accessible location and specify the path to the JSON file on the service_account_json property in the configuration.yaml file of the agent.

For more information about service accounts, roles, and permissions, see:

SAP HANA metrics in Cloud Monitoring

The monitoring agent for SAP HANA sends the data that it retrieves from SAP HANA to Monitoring as custom metric data. To send the data, the agent uses the Cloud Monitoring API, which is enabled by default when you create a Cloud project.

For an overview of Monitoring, see Introduction to Cloud Monitoring.

Finding the SAP HANA data in Cloud Monitoring

The metric data from SAP HANA is available to view as soon as Monitoring receives it.

To view your SAP HANA data in Monitoring, create a chart in the Cloud Console by using either the dashboard editor or the Metrics Explorer tool and enter sap_hana in the metric field. You can then select your data by metric type from the drop-down list of all available metric types that include sap_hana.

In Monitoring, to show the data for only a specific SAP HANA instance, you can filter your views by using, for example, the name of the SAP HANA host VM, which is a metric label in Monitoring.

For more information, see:

Dashboard definition for SAP HANA on GitHub

On GitHub, you can find a custom dashboard definition for SAP HANA that you can install in your Google Cloud project. See the GoogleCloudPlatform/monitoring-dashboard-samples repository on GitHub. The SAP HANA dashboard definition file, performance.json, and a README.md file are in the /dashboards/sap-hana subdirectory.

After you install the custom dashboard for SAP HANA, you can use it as it is, or you can further customize it in the Cloud Console.

To install the custom dashboard for SAP HANA, see Installable dashboards.

Alert notifications

To alert you when a metric reaches a threshold that you define, you can configure alert notifications in Monitoring.

For more information, see Introduction to alerting.

Password management

By default, the monitoring agent for SAP HANA uses the predefined SAP HANA SYSTEM database user account to query the SAP HANA metrics. You must provide the password for the database user account for each SAP HANA database that the agent monitors.

To provide the password to the monitoring agent for SAP HANA securely, you can use Secret Manager, which is a charged service of Google Cloud.

Alternatively, you can specify a database user password in plain text on the password property of each instance definition in the configuration.yaml file.

For information about Secret Manager pricing, see Pricing.

SSL/TLS Support

If a SAP HANA instance uses the Transport Layer Secure (TLS)/Secure Sockets Layer (SSL) protocol to secure communication between the SAP HANA database and clients that use the SAP HANA SQL interface, you need to enable TLS/SSL support in the monitoring agent for SAP HANA by specifying enable_ssl: true in the instance definition in configuration.yaml file.

If SAP HANA is also configured to validate certificates, you can configure the agent to support certificate validation and specify paths and passwords for a keystore and trustore.

For a description of the agent configuration properties for SSL and certificate checking, see Configuration property descriptions.

Logging

When you start the monitoring agent for SAP HANA for the first time, the agent creates a log file, /var/log/google-saphanamonitoring-agent.log, on the local host, where all log entries are written.

By default, when the log file reaches 10.5 MB, older logs are discarded as new logs are appended to the file.

Logging for the agent is configured by the conf/logging.properties file.

Support

For issues with Google Cloud infrastructure or services, contact Customer Care. You can find contact information on the Support Overview page in the Google Cloud Console. If Customer Care determines that a problem resides in your SAP systems, you are referred to SAP Support.

For SAP product-related issues, log your support request with SAP support. SAP evaluates the support ticket and, if it appears to be a Google Cloud infrastructure issue, transfers the ticket to the Google Cloud component BC-OP-LNX-GOOGLE or BC-OP-NT-GOOGLE.

Support requirements

Before you can receive support for SAP systems and the Google Cloud infrastructure and services that they use, you must meet the minimum support plan requirements.

For more information about the minimum support requirements for SAP on Google Cloud, see:

Defining custom queries

You define custom queries in the configuration.yaml file.

A custom query definition consists of property values that describe the query and the returned metrics, as well as an SQL statement that retrieves the data from SAP HANA.

For a full list of the system views that SAP HANA makes available for querying, see the SAP HANA SQL and System Views Reference.

When you create a custom query, keep in mind that Monitoring works best for summary or aggregate data and is not intended to store high-cardinality data points. Use SQL aggregate functions like SUM, AVG, COUNT, MIN and MAX to limit the number of records returned.

For an example of a custom query, see Example custom queries.

Custom query properties

The following table describes the properties that you use to define a custom query.

Property Value Description
sample_interval Int At the top of the configuration.yaml file, defines the default interval in seconds between between queries. In the definition of a custom query, the property defines the sample interval for that query only, overriding the default.
query_timeout Int Time allowed, in seconds, for the execution of the query before the query fails. The default is 300 seconds (5 minutes).
queries The following property descriptions apply only to custom queries.
- name String The name of the query.
enabled Boolean Determines whether the agent submits the query to SAP HANA. Valid values are true or false.
description String Your description of the metrics that the query retrieves.
sql String An SQL statement that you specify that the agent issues to SAP HANA. SQL statements must conform to the SQL syntax that is defined by SAP. For more information, see SAP HANA SQL and System Views Reference.
columns Each column definition describes a metric for handling by Monitoring.
- name String The name of the metric, which defines the metric type in Monitoring.
metric_type String For handling by Monitoring, the type of metric, GAUGE or CUMULATIVE.
value_type String For handling by Monitoring, the data type of the metric, BOOL, INT64, or DOUBLE. Specify a data type that is supported by the metric type. For more information, see Supported combinations.
description String Description of the metric.
units String The units in which the metric value is reported. For more information, see the unit field documentation.

Example custom queries

The following example shows two custom queries. These queries are also included in the configuration.yaml file that is created in the /usr/sap/google-saphanamonitoring-agent/conf/ directory when you install the monitoring agent for SAP HANA. The example queries are intended for you to use as models for your own custom queries.

- name: custom_memory_utilization
  enabled: false
  # sample_interval: 120
  description: |
    Custom Total memory utilization by services
  sql: |
    SELECT
           SUM(TOTAL_MEMORY_USED_SIZE) AS "mem_used",
           SUM(PHYSICAL_MEMORY_SIZE) AS "resident_mem_used"
    FROM M_SERVICE_MEMORY;
  columns:
  - name: mem_used
    metric_type: GAUGE
    value_type: INT64
    description: |
      Amount of memory from the memory pool.
    units: By
  - name: resident_mem_used
    metric_type: GAUGE
    value_type: INT64
    description: |
      Amount of memory used in total by all the services.
    units: By
- name: custom_total_memory_usage
  enabled: false
  # sample_interval: 120
  description: |
    Custom Amount of memory used in total by all column-tables
  sql: |
    SELECT
           SUM(MEMORY_SIZE_IN_TOTAL) AS "table_mem_used"
    FROM M_CS_TABLES;
  columns:
  - name: table_mem_used
    metric_type: GAUGE
    value_type: INT64
    description: |
      Amount of memory used in total (sum of memory size in the main, delta, and history
      parts) by all column-tables.
    units: By
# Add additional queries for custom metrics

Default query reference

The built-in default queries of the monitoring agent for SAP HANA retrieve the following information from SAP HANA.

If you don't need the metrics provided by a particular default query, you can disable the query by specifying enabled: false after the query name in the configuration.yaml file.

default_column_queries

The built-in query default_column_queries retrieves information about the usage of memory by columns from SAP HANA M_CS_ALL_COLUMNS system view.

The query sends the following metrics to Monitoring:

Metric name Metric Description
Column memory size column/memory/total_size Specifies the sum of the MEMORY_SIZE_IN_MAIN and MEMORY_SIZE_IN_DELTA columns.

default_component_queries

The built-in query default_component_queries retrieves service-specific memory usage by logical component from SAP HANA. The query uses the SAP HANA M_SERVICE_COMPONENT_MEMORY system view.

The query sends the following metrics to Monitoring:

Metric name Metric Description
Component memory used component/memory/total_used_size Amount of memory that is currently used for the logical component.

default_connection_query

The built-in query default_connection_query retrieves the number of running and idle connections from SAP HANA by using the SAP HANA M_CONNECTIONS system view.

The query sends the following metrics to Monitoring:

Metric name Metric Description
System connections system/connection/total The number of idle and running connections of the system.

default_cpu_queries

The built-in query default_cpu_queries retrieves cpu usage information from SAP HANA by using the SAP HANA M_HOST_RESOURCE_UTILIZATION system view.

The query sends the following metrics to Monitoring:

Metric name Metric Description
CPU usage time host/cpu/usage_time CPU time spent.

default_alert_query

The built-in query default_alert_query retrieves the number and rating of current alerts from the SAP HANA Statistics Server by using the SAP HANA STATISTICS_CURRENT_ALERTS view of the _SYS_STATISTICS schema.

The query sends the following metrics to Monitoring:

Metric name Metric Description
Alerts count system/alert/total The number of current alerts from the system statistics: "1", "2", "3", "4", or "5".

default_host_queries

The built-in query default_host_queries retrieves memory usage by the host from SAP HANA. The query uses the SAP HANA M_HOST_RESOURCE_UTILIZATION system view.

The query sends the following metrics to Monitoring:

Metric name Metric Description
Host memory size host/memory/total_size Total physical memory on the host.
Host memory used host/memory/total_used_size Used physical memory on the host.
Swap space size host/swap_space/total_size Total swap memory on the host.
Swap space used host/swap_space/total_used_size Used swap memory on the host.
Instance memory used host/instance_memory/total_used_size Amount of memory from the memory pool that is currently in actual use by SAP HANA processes.
Peak instance memory used host/instance_memory/total_peak_used_size Peak memory from the memory pool that was in use by SAP HANA processes since start of the instance (sample based value).
Instance memory allocated host/instance_memory/total_allocated_size Size of the memory pool for all SAP HANA processes.
Instance code size host/instance_code/total_size Code size, including shared libraries of SAP HANA processes.
Instance shared memory allocated host/instance_shared_memory/total_allocated_size Shared memory size of SAP HANA processes.

default_replication_query

The built-in query default_replication_query retrieves replication latency information from SAP HANA by using the SAP HANA M_SERVICE_REPLICATION system view.

The query sends the following metrics to Monitoring:

Metric name Metric Description
Replication data latency system/replication_data_latency/total_time The data replication latency of the system.

default_rowstore_query

The built-in query default_rowstore_query retrieves the size of the rowstore from SAP HANA by using the SAP HANA M_RS_TABLES system view.

The query sends the following metrics to Monitoring:

Metric name Metric Description
Rowstore memory size rowstore/memory/total_size The rowstore memory size consisting of both the fixed and variable parts.

default_schema_type_queries

The built-in query default_schema_type_queries retrieves memory usage and record count information from SAP HANA by using the SAP HANA M_CS_TABLES system view.

The query sends the following metrics to Monitoring:

Metric name Metric Description
Schema memory size schema/memory/total_size Specifies the total memory size as the sum of memory size in the main, delta, and history parts.
Schema record count schema/record/total Specifies the record count.

default_schema_queries

The built-in query default_schema_queries retrieves schema usage information from SAP HANA by using the SAP HANA M_CS_TABLES system view.

The query sends the following metrics to Monitoring:

Metric name Metric Description
Schema estimated max memory size schema/memory/estimated_max_total_size Specifies the estimated maximum memory consumption, in total, for the fully loaded table (data for open transactions is not included).
Schema last compressed records count schema/record/last_compressed_total Specifies the number of entries in main during the last optimize compression run.
Schema read count schema/read/total_count Specifies the number of read accesses on the table or partition. This is not the number of SELECT statements against this table. A SELECT statement may involve several read accesses.
Schema write count schema/write/total_count Specifies the number of write accesses on the table or partition. This is not the number of DML and DDL statements against this table. A DML or DDL statement may involve several write accesses.
Schema merge count schema/merge/total_count Specifies the number of delta merges done on the table or partition.

default_service_queries

The built-in query default_service_queries retrieves information from SAP HANA about the memory used by services. The query uses the SAP HANA M_SERVICE_MEMORY system view.

The query sends the following metrics to Monitoring:

Metric name Metric Description
Service memory used service/memory/total_used_size Specifies the amount of memory in use from the memory pool.
Service logical memory size service/logical_memory/total_size Specifies the virtual memory size from the operating system perspective.
Service physical memory size service/physical_memory/total_size Specifies the physical resident memory size from the operating system perspective.
Service code size service/code/total_size Specifies the code size, including shared libraries.
Service stack size service/stack/total_size Specifies the stack size.
Service heap memory allocated service/heap_memory/total_allocated_size Specifies the heap part of the memory pool.
Service heap memory used service/heap_memory/total_used_size Specifies the amount of pool heap memory that is in use.
Service shared memory allocated service/shared_memory/total_allocated_size Specifies the shared memory part of the memory pool.
Service shared memory used service/shared_memory/total_used_size Specifies the amount of pool shared memory that is in use.
Service compactors allocated size service/compactor/total_allocated_size Specifies the part of the memory pool that can potentially (if unpinned) be freed during a memory shortage.
Service compactors freeable size service/compactors/total_freeable_size pecifies the memory that can be freed during a memory shortage.
Service memory allocation limit service/memory/allocation_limit Specifies the maximum memory pool size (configurable value).
Service memory effective allocation limit service/memory/effective_allocation_limit Specifies the effective maximum memory pool size, considering the pool sizes of other processes (computed value).

default_transaction_query

The built-in query default_transaction_query retrieves the total number of transactions (update, commit, and rollback) from SAP HANA by using the SAP HANA M_WORKLOAD system view.

The query sends the following metrics to Monitoring:

Metric name Metric Description
System transactions system/transaction/total_count The number of update, commit, and rollback transactions.

What's next