SAP HANA on Bare Metal Solution deployment guide

This guide covers the aspects of setting up a Bare Metal Solution environment that are unique or relevant to SAP HANA.

In a Bare Metal Solution environment, you install SAP HANA and other SAP software yourself by following the SAP documentation.

This guide also describes the Google Cloud foundational services for SAP that are required or optional for SAP HANA in a Bare Metal Solution environment.

For an overview of SAP HANA on Bare Metal Solution to help you plan for your deployment, see SAP HANA on Bare Metal Solution planning guide.

High-level steps

The following list shows the high-level steps that you need to complete to deploy SAP HANA on Bare Metal Solution after Google Cloud notifies you that your machines are ready.

Complete the connection to your new machines by following the Bare Metal Solution documentation. For the set up procedures, see Setting up the Bare Metal Solution environment in the Bare Metal Solution documentation.
Validate the machine configuration for SAP HANA.
Create IAM service accounts for the applications that need to access Google Cloud resources. See Set up your service accounts.
Establish a connection to the internet for software updates. See Accessing the internet in the Bare Metal Solution documentation.
Establish connection to Google Cloud APIs. See Setting up access to Google Cloud APIs and services in the Bare Metal Solution documentation.
Install Google Cloud's Agent for SAP.
Set up access to an Network Time Protocol (NTP) server, which is not covered in this guide.
Set up a Domain Name System (DNS) server, which is not covered in this guide.
Installing the Cloud Storage Backint agent for SAP HANA, which is optional.

Validating the machine configuration for SAP HANA

When you log into your machines for the first time, you need to confirm that the machines, operating systems, and storage that you received are what you ordered and that everything is configured correctly.

In addition to the general validation checks that are listed in the Bare Metal Solution documentation, validate the following items that are specific to SAP HANA:

That your operating system is the correct, SAP-specific version
That the SAP HANA volumes and file system mounts are mapped correctly
That the storage is configured correctly
That the time is synchronized between the servers

For information about Bare Metal Solution machine validation, see Logging in to a Bare Metal Solution machine for the first time.

Set up application service accounts

Any application program, agent, or process that accesses Google Cloud resources from a Bare Metal Solution environment needs an IAM service account for authentication and authorization.

To create a service account:

In the Google Cloud console, go to the Service accounts page.

Go to the Service Accounts page
Select your Google Cloud project.
Click Create Service Account.
Specify a name for the service account and, optionally, a description.
Click Create and Continue.
On the Grant this service account access to project panel, select the roles that contain the permissions that your application needs.

For example, the fence_gce agent that is used in high-availability clusters needs the baremetalsolution.service.resetInstance permission, which is included in the IAM roles roles/baremetalsolution.editor and roles/baremetalsolution.admin.
Click Continue.
As appropriate, grant other users access to the service account.
Click Done.
On the Service accounts page in the Google Cloud console, click the email address of the service account that you just created.
Under the service account name, click the Keys tab.
Click the Add Key drop-down menu, and then select Create new key to create a service account key.
Accept JSON as the key type and click Create. A private key is saved to your computer.
Upload the JSON key file to a secure location on the Bare Metal Solution servers that is accessible to the applications that require it.
Configure your applications to access the JSON key file.

For example, when you create the high-availability cluster resource that uses the fence_gce agent, you specify the path and file name of the JSON key file in the fence_gce configuration properties.

For more information about IAM service accounts, see:

Confirming connectivity to the application servers

Confirm that you have network connectivity between the SAP HANA servers and the application servers.

Make sure that your Google Cloud and guest operating system firewall rules allow traffic between the servers.

Validating the volume-to-storage mapping

The SAP HANA volumes are mapped to storage for you already, but you should validate it, just to be sure.

With 48 TiB of storage, you should see the following volume layout for SAP HANA on a Bare Metal Solution server:

/hana/log: Four 768 GiB LUNS, striped with LVM by using a 64 KiB block size
/hana/data: Four 10.5 TiB LUNS, striped with LVM by using a 256 KiB block size
/hana/shared: A 3 TiB volume
/usr/sap: A 200 GiB volume shared with the operating system

The following table describes the configuration of the volume groups and logical volumes.

Volume group	Logical volume	Configuration
`hanadatavg`	`data`	Striped across 4 devices of at least 10 TiB each, with a block size of 256 KiB
`hanalogvg`	`log`	Striped across 4 devices of at least 768 GiB each, with a block size of 64 KiB
`hanasharedvg`	`shared`	Linear (not striped) on a 3 TiB device

Installing SAP HANA on Bare Metal Solution

To install SAP HANA on a Bare Metal Solution host machine, you need to follow the SAP installation documentation for your version of SAP HANA.

To find the installation documentation for your version of SAP HANA, see the SAP Help Portal.

Configure a high-availability cluster

Configure clustering or a suitable high availability solution for your business critical workloads. We recommend Pacemaker clustering.

With the exception of a few requirements and recommendations, configure high-availability clusters on Bare Metal Solution machines by following the clustering instructions that your operating system vendor provides.

For more information about the Google Cloud requirements and recommendations for configuring a highly available SAP HANA system on Bare Metal Solution, see High-availability clusters for SAP HANA on Bare Metal Solution.

Floating IP address

A high-availability cluster uses a floating or virtual IP address (VIP) to direct network communication to the active SAP HANA system. When configuring your VIP, use the standard IPaddr2 Pacemaker resource. The following example shows an IPaddr2 resource definition on SUSE:

crm configure primitive rsc_ip_HA1_HDB10 IPaddr2 \
    params ip=10.0.0.79 \
    op monitor interval=10 timeout=20

Configure the IPaddr2 resource with a colocation relationship with the HANA database so that it moves with the active database on a failover. For example:

colocation col_saphana_ip_HA1_HDB10 2000: rsc_ip_HA1_HDB10:Started \
   msl_SAPHana_HA1_HDB10:Master

Configure a fencing device

A Linux high-availability cluster for SAP HANA requires a fencing device that can reset failed nodes to avoid split-brain scenarios.

In a Bare Metal Solution environment, you have two options for implementing a fencing device:

SBD with a software watchdog and shared block storage. SBD is recommended, but not all operating systems support a software watchdog with SBD.
The fence_gce agent and the resetInstance method of the Bare Metal Solution API.

Configure SBD fencing with shared block storage

To configure SBD fencing, you need to follow the instructions that are provided by your software vendor.

This section shows you only how to identify and validate the storage devices that Google Cloud provisioned for SBD fencing.

To configure SBD fencing on SUSE, see:

Validate the block storage devices for SBD fencing

Google Cloud creates three shared LUNs for SBD block storage when it provisions an order for a high-availability cluster that uses SBD fencing.

You can identify the shared storage devices for SBD by issuing the multipath -ll command to list the multipath devices on both servers:

multipath -ll | grep "size=Size of your LUN" -B1

In each list, look for three devices of the same size and make a note of their names. The default size is 50 GB.

In the following example, the string size=50G is used to filter the list of multipath devices that is returned by the multipath -ll command:

multipath -ll | grep "size=50G" -B1

The example output from the preceding command includes the three 50 GB multipath devices to be used for SBD fencing:

3600a098038314566785d5150506a594b dm-16,LUN C-Mode
size=50G features='3 queue_if_no_path pg_init_retries 50' hwhandler='1 alua' wp=rw
--
3600a098038314566785d5150506a594c dm-15,LUN C-Mode
size=50G features='3 queue_if_no_path pg_init_retries 50' hwhandler='1 alua' wp=rw
--
3600a098038314566785d5150506a594d dm-11,LUN C-Mode
size=50G features='3 queue_if_no_path pg_init_retries 50' hwhandler='1 alua' wp=rw

The names of the three devices must be exactly the same on both servers, which indicates that they are shared LUNs.

When you follow the SBD configuration instructions that are provided by your operating system vendor, you reference these three device names.

For example, when you initialize each device for SBD use and set the wait and timeout values, you specify the device names as shown in the following example:

sbd -d /dev/mapper/3600a098038314566785d5150506a594b -1 60 -4 180 create

When you set the SBD configuration in /etc/sysconfig/sbd, you also need to reference the device names. For example:

SBD_DEVICE="/dev/mapper/3600a098038314566785d5150506a594b;/dev/mapper/3600a098038314566785d5150506a594c;/dev/mapper/3600a098038314566785d5150506a594d"
[..]
SBD_PACEMAKER="yes"
[...]
SBD_STARTMODE="always"

Configure the `fence_gce` agent

The following high-level steps help you configure a fencing resource for your high availability cluster that uses the fence_gce agent that is included with your Linux operating system.

In your Google Cloud project, assign an IAM role that includes the baremetalsolution.service.resetInstance permission to a new or existing IAM service account for which you have the JSON key file.

The predefined role roles/baremetalsolution.editor includes the required permission or you can create a custom role for more granular control over the permissions that the role includes.

For more information, see:

Upload the service account JSON key file to all Bare Metal Solution servers in the high-availability cluster.
Confirm that your version of the fence_gce agent supports Bare Metal Solution.
1. SSH into your Bare Metal Solution server
2. Display the fence_gce agent help:
```
$ sudo fence_gce --help
```
3. In the fence_gce help, look for the following properties:
```
--baremetalsolution            Enable on bare metal
...
--serviceaccount=[filename]    Service account json file location e.g.
                                   serviceaccount=/somedir/service_account.json
```
  If you do not see both the --baremetalsolution and --serviceaccount=[filename] properties, you need to update the agent. Contact Cloud Customer Care for patching instructions. The patching procedure differs depending on which Linux version you are using.

Specify the fence_gce agent as your fencing device on each cluster node when you configure the cluster resources according to the clustering instructions that are provided by your operating system vendor.

When you define the resource, specify the following fence_gce parameters to enable the agent for the Bare Metal Solution environment:

baremetalsolution="true"
serviceaccount="file_path/key_file.json"

The following example shows a fence_gce fencing device resource that is defined on SUSE:

crm configure primitive STONITH-"server_name" stonith:fence_gce \
 op monitor interval="300s" timeout="60s" on-fail="restart" \
 op start interval="0" timeout="60s" onfail="restart" \
 params port="server_name" zone="Google_Cloud_zone" project="Google_Cloud_project_ID" \
   baremetalsolution="true" method="cycle" \
   serviceaccount="file_path/key_file.json"

The fencing resource example uses the following definitions:

Definition	Explanation
`baremetalsolution=true`	Configures the fencing agent for operation in the Bare Metal Solution environment.
`serviceaccount=file_path/key_file.json`	The file path to the JSON key file that contains the key for the service account that the fencing agent uses to access the Google Cloud APIs.
`port=server_name`	The original host name of the Bare Metal Solution server that the fencing device can reset.
`method=cycle`	Defines the method to use for resetting the failed server.

After you define each fencing resource, set the location of the fencing device to a server other than the server that you intend it to reset:
```
crm configure location LOC_STONITH_server_name STONITH-server_name -inf: server_name
```

Install Google Cloud's Agent for SAP

Google Cloud's Agent for SAP is required for SAP support of SAP systems running on a Bare Metal Solution server.

If you use the following RHEL or SLES "for SAP" OS images that Google Cloud provides, then Google Cloud's Agent for SAP is packaged with the OS images:

RHEL: all "for SAP" images
SLES: SLES 15 SP4 for SAP and later versions

For information about the operating systems available for running SAP on Bare Metal Solution servers, see the following:

For SAP HANA, see Certified operating systems.
For SAP NetWeaver, see Certified operating systems.

To install Google Cloud's Agent for SAP on a Bare Metal Solution server, follow these steps:

Establish an SSH connection to your Bare Metal Solution server.

In your terminal, install the agent by running the command that is specific to your operating system:

RHEL

sudo tee /etc/yum.repos.d/google-cloud-sap-agent.repo << EOM
[google-cloud-sap-agent]
name=Google Cloud Agent for SAP
baseurl=https://packages.cloud.google.com/yum/repos/google-cloud-sap-agent-el$(cat /etc/redhat-release | cut -d . -f 1 | tr -d -c 0-9)-\$basearch
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
EOM
sudo yum install google-cloud-sap-agent

SLES 15

sudo zypper addrepo --refresh https://packages.cloud.google.com/yum/repos/google-cloud-sap-agent-sles15-\$basearch google-cloud-sap-agent
sudo zypper install google-cloud-sap-agent

SLES 12

sudo zypper addrepo --refresh https://packages.cloud.google.com/yum/repos/google-cloud-sap-agent-sles12-\$basearch google-cloud-sap-agent
sudo zypper install google-cloud-sap-agent

Open the following configuration file:

/etc/google-cloud-sap-agent/configuration.json

Modify or add the following properties to the configuration file:
- In the first section:
  - bare_metal: set to true.
- Add the cloud_properties section and specify the following properties:
  - project_id: specify your Cloud project ID.
  - instance_id: specify the instance ID of the Bare Metal Solution server.
  - region: specify the region of the Bare Metal Solution server.
Save the configuration file.

Restart the agent:

sudo systemctl restart google-cloud-sap-agent

For more information about the configuration properties, see Install Google Cloud's Agent for SAP on a Bare Metal Solution server

After you install Google Cloud's Agent for SAP and the SAP Host Agent, validate the installation of Google Cloud's Agent for SAP, as described in Validate your installation of the agent.

Validate your installation of Google Cloud's Agent for SAP

Establish an SSH connection with your Bare Metal Solution server.

Run the following command:

systemctl status google-cloud-sap-agent

If the agent is functioning properly, then the output contains active (running). For example:

google-cloud-sap-agent.service - Google Cloud Agent for SAP
Loaded: loaded (/usr/lib/systemd/system/google-cloud-sap-agent.service; enabled; vendor preset: disabled)
Active:  active (running)  since Fri 2022-12-02 07:21:42 UTC; 4 days ago
Main PID: 1337673 (google-cloud-sa)
Tasks: 9 (limit: 100427)
Memory: 22.4 M (max: 1.0G limit: 1.0G)
CGroup: /system.slice/google-cloud-sap-agent.service
       └─1337673 /usr/bin/google-cloud-sap-agent

If the agent isn't running, then restart the agent.

Verify that SAP Host Agent is receiving metrics

To verify that the infrastructure metrics are collected by Google Cloud's Agent for SAP and sent correctly to the SAP Host Agent, follow these steps:

In your SAP system, enter transaction ST06.
In the overview pane, check the availability and content of the following fields for the correct end-to-end setup of the SAP and Google monitoring infrastructure:
- Cloud Provider: Google Cloud Platform
- Enhanced Monitoring Access: TRUE
- Enhanced Monitoring Details: ACTIVE

Set up monitoring for SAP HANA

Optionally, you can monitor your SAP HANA instances using Google Cloud's Agent for SAP. From version 2.0, you can configure the agent to collect the SAP HANA monitoring metrics and send them to Cloud Monitoring. Cloud Monitoring allows you to create dashboards to visualize these metrics, set up alerts based on metric thresholds, and more.

To monitor SAP instances running in a Bare Metal Solution environment you can install Google Cloud's Agent for SAP either on a Bare Metal Solution server or a Compute Engine VM instance. Depending on where the agent's host is placed in your deployment architecture, you can enable a local, remote, or hybrid collection of the SAP HANA monitoring metrics.

For more information about the collection of SAP HANA monitoring metrics using Google Cloud's Agent for SAP, see SAP HANA monitoring metrics collection.

Installing the Backint agent

If you choose to use the Backint agent, install it on the SAP HANA host machine the same way that you would in an on-premises environment.

As described in the Backint agent installation guide, you need to create a service account on Google Cloud and provide the service account key to the Backint agent on the SAP HANA host machine.

For installations instructions, see Cloud Storage Backint agent for SAP HANA installation guide.

Maintaining your Bare Metal Solution environment

For your Bare Metal Solution environments that run SAP HANA, Google Cloud provides the following self-service functionalities that you can use to maintain your Bare Metal Solution servers, storage, and networks:

Manage servers:
Manage storage:
- View storage details
- Organize storage volumes using labels
Manage NFS file storage:
- View NFS file storage details
- Organize NFS volumes using labels
Managing networks:
Configure the interactive serial console

Getting support for SAP HANA on Bare Metal Solution

If you need help resolving a problem with SAP HANA deployments on Bare Metal Solution, gather the required diagnostic information and contact Cloud Customer Care. For more information, see SAP on Bare Metal Solution diagnostic information.

Support

For issues with Google Cloud infrastructure or services, contact Customer Care. You can find contact information on the Support Overview page in the Google Cloud console. If Customer Care determines that a problem resides in your SAP systems, you are referred to SAP Support.

For SAP product-related issues, log your support request with SAP support. SAP evaluates the support ticket and, if it appears to be a Google Cloud infrastructure issue, transfers the ticket to the Google Cloud component BC-OP-LNX-GOOGLE or BC-OP-NT-GOOGLE.

Support requirements

Before you can receive support for SAP systems and the Google Cloud infrastructure and services that they use, you must meet the minimum support plan requirements.

For more information about the minimum support requirements for SAP on Google Cloud, see:

Getting support for SAP on Google Cloud
SAP Note 2456406 - SAP on Google Cloud Platform: Support Prerequisites (An SAP user account is required)