This guide covers the aspects of setting up a Bare Metal Solution environment that are unique or relevant to SAP HANA.
In a Bare Metal Solution environment, you install SAP HANA and other SAP software yourself by following the SAP documentation.
This guide also describes the Google Cloud foundational services for SAP that are required or optional for SAP HANA in a Bare Metal Solution environment.
For an overview of SAP HANA on Bare Metal Solution to help you plan for your deployment, see SAP HANA on Bare Metal Solution planning guide.
The following list shows the high-level steps that you need to complete to deploy SAP HANA on Bare Metal Solution after Google Cloud notifies you that your machines are ready.
- Complete the connection to your new machines by following the Bare Metal Solution documentation. For the set up procedures, see Setting up the Bare Metal Solution environment in the Bare Metal Solution documentation.
- Validate the machine configuration for SAP HANA.
- Create IAM service accounts for the applications that need to access Google Cloud resources. See Set up your service accounts.
- Establish a connection to the internet for software updates. See Accessing the internet in the Bare Metal Solution documentation.
- Establish connection to Google Cloud APIs. See Setting up access to Google Cloud APIs and services in the Bare Metal Solution documentation.
- Set up access to an Network Time Protocol (NTP) server, which is not covered in this guide.
- Set up a Domain Name System (DNS) server, which is not covered in this guide.
- Installing the Cloud Storage Backint agent for SAP HANA, which is optional.
- Installing the Google Cloud monitoring agent for SAP HANA, which is optional.
Validating the machine configuration for SAP HANA
When you log into your machines for the first time, you need to confirm that the machines, operating systems, and storage that you received are what you ordered and that everything is configured correctly.
In addition to the general validation checks that are listed in the Bare Metal Solution documentation, validate the following items that are specific to SAP HANA:
- That your operating system is the correct, SAP-specific version
- That the SAP HANA volumes and file system mounts are mapped correctly
- That the storage is configured correctly
- That the time is synchronized between the servers
For information about Bare Metal Solution machine validation, see Logging in to a Bare Metal Solution machine for the first time.
Set up application service accounts
Any application program, agent, or process that accesses Google Cloud resources from a Bare Metal Solution environment needs an IAM service account for authentication and authorization.
To create a service account:
In the Google Cloud console, go to the Service accounts page.
Select your Google Cloud project.
Click Create Service Account.
Specify a name for the service account and, optionally, a description.
Click Create and Continue.
On the Grant this service account access to project panel, select the roles that contain the permissions that your application needs.
For example, the
fence_gceagent that is used in high-availability clusters needs the
baremetalsolution.service.resetInstancepermission, which is included in the IAM roles
As appropriate, grant other users access to the service account.
On the Service accounts page in the Google Cloud console, click the email address of the service account that you just created.
Under the service account name, click the Keys tab.
Click the Add Key drop-down menu, and then select Create new key to create a service account key.
Accept JSON as the key type and click Create. A private key is saved to your computer.
Upload the JSON key file to a secure location on the Bare Metal Solution servers that is accessible to the applications that require it.
Configure your applications to access the JSON key file.
For example, when you create the high-availability cluster resource that uses the
fence_gceagent, you specify the path and file name of the JSON key file in the
For more information about IAM service accounts, see:
Confirming connectivity to the application servers
Confirm that you have network connectivity between the SAP HANA servers and the application servers.
Make sure that your Google Cloud and guest operating system firewall rules allow traffic between the servers.
Validating the volume-to-storage mapping
The SAP HANA volumes are mapped to storage for you already, but you should validate it, just to be sure.
With 48 TiB of storage, you should see the following volume layout for SAP HANA on a Bare Metal Solution server:
/hana/log: Four 768 GiB LUNS, striped with LVM by using a 64 KiB block size
/hana/data: Four 10.5 TiB LUNS, striped with LVM by using a 256 KiB block size
/hana/shared: A 3 TiB volume
/usr/sap: A 200 GiB volume shared with the operating system
The following table describes the configuration of the volume groups and logical volumes.
|Volume group||Logical volume||Configuration|
||Striped across 4 devices of at least 10 TiB each, with a block size of 256 KiB|
||Striped across 4 devices of at least 768 GiB each, with a block size of 64 KiB|
||Linear (not striped) on a 3 TiB device|
Installing SAP HANA on Bare Metal Solution
To install SAP HANA on a Bare Metal Solution host machine, you need to follow the SAP installation documentation for your version of SAP HANA.
To find the installation documentation for your version of SAP HANA, see the SAP Help Portal.
Configure a high-availability cluster
Configure clustering or a suitable high availability solution for your business critical workloads. We recommend Pacemaker clustering.
With the exception of a few requirements and recommendations, configure high-availability clusters on Bare Metal Solution machines by following the clustering instructions that your operating system vendor provides.
For more information about the Google Cloud requirements and recommendations for configuring a highly available SAP HANA system on Bare Metal Solution, see High-availability clusters for SAP HANA on Bare Metal Solution.
Floating IP address
A high-availability cluster uses a floating or virtual IP address (VIP) to
direct network communication to the active SAP HANA system. When configuring
your VIP, use the standard
IPaddr2 Pacemaker resource. The following
example shows an
IPaddr2 resource definition on SUSE:
crm configure primitive rsc_ip_HA1_HDB10 IPaddr2 \ params ip=10.0.0.79 \ op monitor interval=10 timeout=20
IPaddr2 resource with a colocation relationship with the
HANA database so that it moves with the active database on a failover.
colocation col_saphana_ip_HA1_HDB10 2000: rsc_ip_HA1_HDB10:Started \ msl_SAPHana_HA1_HDB10:Master
Configure a fencing device
A Linux high-availability cluster for SAP HANA requires a fencing device that can reset failed nodes to avoid split-brain scenarios.
In a Bare Metal Solution environment, you have two options for implementing a fencing device:
- SBD with a software watchdog and shared block storage. SBD is recommended, but not all operating systems support a software watchdog with SBD.
fence_gceagent and the
resetInstancemethod of the Bare Metal Solution API.
Configure SBD fencing with shared block storage
To configure SBD fencing, you need to follow the instructions that are provided by your software vendor.
This section shows you only how to identify and validate the storage devices that Google Cloud provisioned for SBD fencing.
To configure SBD fencing on SUSE, see:
- Storage Protection and SBD on SUSE Linux Enterprise HAE 12 SP4
- Storage Protection and SBD on SUSE Linux Enterprise HAE 12 SP5
- Storage Protection and SBD on SUSE Linux Enterprise HAE 15 SP1
- Storage Protection and SBD on SUSE Linux Enterprise HAE 15 SP2
Validate the block storage devices for SBD fencing
Google Cloud creates three shared LUNs for SBD block storage when it provisions an order for a high-availability cluster that uses SBD fencing.
You can identify the shared storage devices for SBD by issuing the
multipath -ll command to list the multipath devices on both servers:
multipath -ll | grep "size=Size of your LUN" -B1
In each list, look for three devices of the same size and make a note of their names. The default size is 50 GB.
In the following example, the string
size=50G is used to filter the
list of multipath devices that is returned by the
multipath -ll command:
multipath -ll | grep "size=50G" -B1
The example output from the preceding command includes the three 50 GB multipath devices to be used for SBD fencing:
3600a098038314566785d5150506a594b dm-16,LUN C-Mode size=50G features='3 queue_if_no_path pg_init_retries 50' hwhandler='1 alua' wp=rw -- 3600a098038314566785d5150506a594c dm-15,LUN C-Mode size=50G features='3 queue_if_no_path pg_init_retries 50' hwhandler='1 alua' wp=rw -- 3600a098038314566785d5150506a594d dm-11,LUN C-Mode size=50G features='3 queue_if_no_path pg_init_retries 50' hwhandler='1 alua' wp=rw
The names of the three devices must be exactly the same on both servers, which indicates that they are shared LUNs.
When you follow the SBD configuration instructions that are provided by your operating system vendor, you reference these three device names.
For example, when you initialize each device for SBD use and set the wait and timeout values, you specify the device names as shown in the following example:
sbd -d /dev/mapper/3600a098038314566785d5150506a594b -1 60 -4 180 create
When you set the SBD configuration in
/etc/sysconfig/sbd, you also need to
reference the device names. For example:
SBD_DEVICE="/dev/mapper/3600a098038314566785d5150506a594b;/dev/mapper/3600a098038314566785d5150506a594c;/dev/mapper/3600a098038314566785d5150506a594d" [..] SBD_PACEMAKER="yes" [...] SBD_STARTMODE="always"
The following high-level steps help you configure a fencing
resource for your high availability cluster that uses the
fence_gce agent that is included with your Linux operating system.
In your Google Cloud project, assign an IAM role that includes the
baremetalsolution.service.resetInstancepermission to a new or existing IAM service account for which you have the JSON key file.
The predefined role
roles/baremetalsolution.editorincludes the required permission or you can create a custom role for more granular control over the permissions that the role includes.
For more information, see:
Upload the service account JSON key file to all Bare Metal Solution servers in the high-availability cluster.
Confirm that your version of the
fence_gceagent supports Bare Metal Solution.
SSH into your Bare Metal Solution server
$sudo fence_gce --help
fence_gcehelp, look for the following properties:
--baremetalsolution Enable on bare metal ... --serviceaccount=[filename] Service account json file location e.g. serviceaccount=/somedir/service_account.json
If you do not see both the
--serviceaccount=[filename]properties, you need to update the agent. Contact Cloud Customer Care for patching instructions. The patching procedure differs depending on which Linux version you are using.
fence_gceagent as your fencing device on each cluster node when you configure the cluster resources according to the clustering instructions that are provided by your operating system vendor.
When you define the resource, specify the following
fence_gceparameters to enable the agent for the Bare Metal Solution environment:
The following example shows a
fence_gcefencing device resource that is defined on SUSE:
crm configure primitive STONITH-"server_name" stonith:fence_gce \ op monitor interval="300s" timeout="60s" on-fail="restart" \ op start interval="0" timeout="60s" onfail="restart" \ params port="server_name" zone="Google_Cloud_zone" project="Google_Cloud_project_ID" \ baremetalsolution="true" method="cycle" \ serviceaccount="file_path/key_file.json"
The fencing resource example uses the following definitions:
Configures the fencing agent for operation in the Bare Metal Solution environment.
The file path to the JSON key file that contains the key for the service account that the fencing agent uses to access the Google Cloud APIs.
The original host name of the Bare Metal Solution server that the fencing device can reset.
Defines the method to use for resetting the failed server.
After you define each fencing resource, set the location of the fencing device to a server other than the server that you intend it to reset:
crm configure location LOC_STONITH_server_name STONITH-server_name -inf: server_name
Install Google Cloud's Agent for SAP
Google Cloud's Agent for SAP is required for SAP support of SAP systems running on a Bare Metal Solution server.
To install Google Cloud's Agent for SAP on a Bare Metal Solution server, follow these steps:
Establish an SSH connection to your Bare Metal Solution server.
In your terminal, install the agent by running the command that is specific to your operating system:
sudo tee /etc/yum.repos.d/google-cloud-sap-agent.repo << EOM [google-cloud-sap-agent] name=Google Cloud Agent for SAP baseurl=https://packages.cloud.google.com/yum/repos/google-cloud-sap-agent-el$(cat /etc/redhat-release | cut -d . -f 1 | tr -d -c 0-9)-\$basearch enabled=1 gpgcheck=0 repo_gpgcheck=1 gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg EOM sudo yum install google-cloud-sap-agent
sudo zypper addrepo --gpgcheck-allow-unsigned-package --refresh https://packages.cloud.google.com/yum/repos/google-cloud-sap-agent-sles15-\$basearch google-cloud-sap-agent sudo zypper --no-gpg-checks install google-cloud-sap-agent
sudo zypper addrepo --gpgcheck-allow-unsigned-package --refresh https://packages.cloud.google.com/yum/repos/google-cloud-sap-agent-sles12-\$basearch google-cloud-sap-agent sudo zypper --no-gpg-checks install google-cloud-sap-agent
Open the following configuration file:
Modify or add the following properties to the configuration file:
- In the first section:
bare_metal: set to
- Add the
cloud_propertiessection and specify the following properties:
project_id: specify your Cloud project ID.
instance_id: specify the instance ID of the Bare Metal Solution server.
region: specify the region of the Bare Metal Solution server.
- In the first section:
Save the configuration file.
Restart the agent:
sudo systemctl restart google-cloud-sap-agent
For more information about the configuration properties, see Install Google Cloud's Agent for SAP on a Bare Metal Solution server
After you install Google Cloud's Agent for SAP and the SAP Host Agent, validate the installation of Google Cloud's Agent for SAP, as described in Validate your installation of the agent.
Validate your installation of Google Cloud's Agent for SAP
Establish an SSH connection with your Bare Metal Solution server.
Run the following command:
systemctl status google-cloud-sap-agent
If the agent is functioning properly, then the output contains
active (running). For example:
google-cloud-sap-agent.service - Google Cloud Agent for SAP Loaded: loaded (/usr/lib/systemd/system/google-cloud-sap-agent.service; enabled; vendor preset: disabled) Active: active (running) since Fri 2022-12-02 07:21:42 UTC; 4 days ago Main PID: 1337673 (google-cloud-sa) Tasks: 9 (limit: 100427) Memory: 22.4 M (max: 1.0G limit: 1.0G) CGroup: /system.slice/google-cloud-sap-agent.service └─1337673 /usr/bin/google-cloud-sap-agent
If the agent isn't running, then restart the agent.
Verify that SAP Host Agent is receiving metrics
To verify that the infrastructure metrics are collected by Google Cloud's Agent for SAP and sent correctly to the SAP Host Agent, follow these steps:
- In your SAP system, enter transaction
In the overview pane, check the availability and content of the following fields for the correct end-to-end setup of the SAP and Google monitoring infrastructure:
- Cloud Provider:
Google Cloud Platform
- Enhanced Monitoring Access:
- Enhanced Monitoring Details:
- Cloud Provider:
Installing the Backint agent
If you choose to use the Backint agent, install it on the SAP HANA host machine the same way that you would in an on-premises environment.
As described in the Backint agent installation guide, you need to create a service account on Google Cloud and provide the service account key to the Backint agent on the SAP HANA host machine.
For installations instructions, see Cloud Storage Backint agent for SAP HANA installation guide.
Installing the monitoring agent for SAP HANA
You can install the monitoring agent for SAP HANA in the same Bare Metal Solution environment as SAP HANA or on a Google Cloud Compute Engine VM.
To locally collect the metrics, you install the agent with the SAP HANA instance on the same Bare Metal Solution server or Compute Engine VM. For more information, see Local collection on Compute Engine VM and Bare Metal Solution server.
To remotely collect the metrics, you install the agent on a different host VM than the VMs or servers running the SAP HANA instances. For more information, see Remote collection from a central Compute Engine VM.
You can also install the agent on the same host VM as the SAP HANA instance, and configure the agent to monitor an SAP HANA instance on the Bare Metal Solution server. For more information, see Hybrid collection.
The following list shows the high-level steps that you need to complete to install the monitoring agent for SAP HANA in a Bare Metal Solution environment.
Create an IAM service account for the monitoring agent to access the Google Cloud resources. You need to create the service account with a JSON service account key. See Set up your service accounts for details.
On the Bare Metal Solution server that will host the monitoring agent, upload the JSON key file to a secure location that is accessible to the agent.
Grant the required IAM role to the service account. See Setting the required IAM roles for details.
Establish connection to Google Cloud APIs. See Configuring Private Google Access for on-premises hosts for details.
Optionally, establish a connection to the internet to download the agent dependencies. See Accessing the internet in the Bare Metal Solution documentation.
Ensure that the required version of the Java Runtime Environment (JRE) and SAP HANA connection drivers are available.
- If the server that you are installing the agent on has access to the internet, the agent downloads the files automatically during installation.
- If the server does not have internet access, you can upload the files to the server yourself. See Prerequisites for details.
Install the monitoring agent. See Installing the monitoring agent for SAP HANA for details.
Modify the agent configuration file
configuration.yaml. See Configuring the agent for details.
You need to remove the comment character,
#, from the
project_idlines and specify the project ID of the Google Cloud project that you are using with Bare Metal Solution. The monitoring data is associated with this Google Cloud project for visualization in the Monitoring dashboards.
Specify the path to the JSON service account key file on the
Specify the commonly used configuration properties.
Enable the monitoring agent as a
systemdservice. See Enable the agent for details.
Start the monitoring agent. See Start the agent for details.
Install the SAP HANA custom dashboard from the GitHub repository into Monitoring. See Install the custom dashboard to view your metrics for details.
For information on operating the monitoring agent for SAP HANA, see Updating the agent, Checking the logs, Stopping the agent, and Troubleshooting in the monitoring agent for SAP HANA installation and operation guide.
To install the monitoring agent for SAP HANA on a Compute Engine VM, see monitoring agent for SAP HANA installation and operation guide.
Maintaining your Bare Metal Solution environment
For your Bare Metal Solution environments that run SAP HANA, Google Cloud provides the following self-service functionalities that you can use to maintain your Bare Metal Solution servers, storage, and networks:
- Manage servers:
- Manage storage:
- Manage NFS file storage:
- Managing networks:
- Configure the interactive serial console
Getting support for SAP HANA on Bare Metal Solution
If you need help resolving a problem with SAP HANA deployments on Bare Metal Solution, gather the required diagnostic information and contact Cloud Customer Care. For more information, see SAP on Bare Metal Solution diagnostic information.
For issues with Google Cloud infrastructure or services, contact Customer Care. You can find contact information on the Support Overview page in the Google Cloud console. If Customer Care determines that a problem resides in your SAP systems, you are referred to SAP Support.
For SAP product-related issues, log your support request with SAP support. SAP evaluates the support ticket and, if it appears to be a Google Cloud infrastructure issue, transfers the ticket to the Google Cloud component BC-OP-LNX-GOOGLE or BC-OP-NT-GOOGLE.
Before you can receive support for SAP systems and the Google Cloud infrastructure and services that they use, you must meet the minimum support plan requirements.
For more information about the minimum support requirements for SAP on Google Cloud, see:
- Getting support for SAP on Google Cloud
- SAP Note 2456406 (An SAP user account is required)