This guide shows you how to automate the deployment of SAP HANA in a performance-optimized SUSE Linux Enterprise Server (SLES) high-availability (HA) cluster that uses alias-IP technology to manage the virtual IP (VIP) address.
To deploy the system, you complete a configuration file template and use Cloud Deployment Manager to deploy two SAP HANA systems in an OS-based HA cluster.
One of the SAP HANA systems functions as the primary, active system and the other functions as a secondary, standby system. Each SAP HANA system is deployed on a Compute Engine VM within the same region, ideally in different zones.
The deployed cluster includes the following functions and features:
- The Pacemaker high-availability cluster resource manager.
- A Google Cloud fencing mechanism.
- A virtual IP (VIP) that uses an alias IP implementation
- The SUSE high-availability pattern.
- The SUSE SAPHanaSR resource agent package.
- Synchronous system replication.
- Memory preload.
- Automatic restart of the failed instance as the new secondary instance.
To deploy an HA system that includes a VIP that uses an internal load-balancer implementation instead of an alias IP implementation, use these instructions to quickly deploy your system with the Deployment Manager templates. Then, after the HA cluster is successfully deployed, change the VIP implementation. For more information, see Migrating a VIP in an SAP HANA HA cluster to an internal load balancer.
If you need a scale-out system with standby hosts for SAP HANA automatic host failover, use the SAP HANA Scale-Out System with SAP HANA Host Auto-Failover Deployment Guide instead.
To deploy an SAP HANA system without a Linux high-availability cluster or standby hosts, use the SAP HANA Deployment Guide.
This guide is intended for advanced SAP HANA users who are familiar with Linux high-availability configurations for SAP HANA.
Before you create the SAP HANA high availability cluster, make sure that the following prerequisites are met:
- You or your organization has a Google Cloud account and you have created a project for the SAP HANA deployment. For information about creating Google Cloud accounts and projects, see Setting up your Google account in the SAP HANA Deployment Guide.
- The SAP HANA installation media is stored in a Cloud Storage bucket that is available in your deployment project and region. For information about how to upload SAP HANA installation media to a Cloud Storage bucket, see Downloading SAP HANA in the SAP HANA Deployment Guide.
If you are using VPC internal DNS, the value of the
VmDnsSettingvariable in your project metadata must be either
ZonalPreferredto enable the resolution of the node names across zones. The default setting of
ZonalOnly. For more information, see:
Creating a network
For security purposes, create a new network. You can control who has access by adding firewall rules or by using another access control method.
If your project has a default VPC network, don't use it. Instead, create your own VPC network so that the only firewall rules in effect are those that you create explicitly.
During deployment, VM instances typically require access to the internet to download Google's monitoring agent. If you are using one of the SAP-certified Linux images that are available from Google Cloud, the VM instance also requires access to the internet in order to register the license and to access OS vendor repositories. A configuration with a NAT gateway and with VM network tags supports this access, even if the target VMs do not have external IPs.
To set up networking:
Go to Cloud Shell.
To create a new network in the custom subnetworks mode, run:
gcloud compute networks create NETWORK_NAME --subnet-mode custom
NETWORK_NAMEwith the name of the new network. The network name can contain only lowercase characters, digits, and the dash character (-).
--subnet-mode customto avoid using the default auto mode, which automatically creates a subnet in each Compute Engine region. For more information, see Subnet creation mode.
Create a subnetwork, and specify the region and IP range:
gcloud compute networks subnets create SUBNETWORK_NAME \ --network NETWORK_NAME --region REGION --range RANGE
Replace the following:
SUBNETWORK_NAME: the name of the new subnetwork.
NETWORK_NAME: the name of the network you created in the previous step.
REGION: the region where you want the subnetwork.
RANGE: the IP address range, specified in CIDR format, such as 10.1.0.0/24. If you plan to add more than one subnetwork, assign non-overlapping CIDR IP ranges for each subnetwork in the network. Note that each subnetwork and its internal IP ranges are mapped to a single region.
Optionally, repeat the previous step and add additional subnetworks.
Setting up a NAT gateway
If you need to create one or more VMs without public IP addresses, you need to use network address translation (NAT) to enable the VMs to access the internet. Use Cloud NAT, a Google Cloud distributed, software-defined managed service that lets VMs send outbound packets to the internet and receive any corresponding established inbound response packets. Alternatively, you can set up a separate VM as a NAT gateway.
To create a Cloud NAT instance for your project, see Using Cloud NAT.
After you configure Cloud NAT for your project, your VM instances can securely access the internet without a public IP address.
Adding firewall rules
By default, an implied firewall rule blocks incoming connections from outside your Virtual Private Cloud (VPC) network. To allow incoming connections, set up a firewall rule for your VM. After an incoming connection is established with a VM, traffic is permitted in both directions over that connection.
You can also create a firewall rule to allow external access to specified ports,
or to restrict access between VMs on the same network. If the
VPC network type is used, some additional default rules also
apply, such as the
default-allow-internal rule, which allows connectivity
between VMs on the same network on all ports.
Depending on the IT policy that is applicable to your environment, you might need to isolate or otherwise restrict connectivity to your database host, which you can do by creating firewall rules.
Depending on your scenario, you can create firewall rules to allow access for:
- The default SAP ports that are listed in TCP/IP of All SAP Products.
- Connections from your computer or your corporate network environment to your Compute Engine VM instance. If you are unsure of what IP address to use, talk to your company's network administrator.
- Communication between VMs in the SAP HANA subnetwork, including communication between nodes in an SAP HANA scale-out system or communication between the database server and application servers in a 3-tier architecture. You can enable communication between VMs by creating a firewall rule to allow traffic that originates from within the subnetwork.
- SSH connections to your VM instance, including SSH from the browser.
- Connection to your VM by using a third-party tool in Linux. Create a rule to allow access for the tool through your firewall.
To create a firewall rule:
In the Cloud Console, go to the Firewall rules page.
At the top of the page, click Create firewall rule.
- In the Network field, select the network where your VM is located.
- In the Targets field, specify the resources on Google Cloud that this rule applies to. For example, specify All instances in the network. Or to to limit the rule to specific instances on Google Cloud, enter tags in Specified target tags.
- In the Source filter field, select one of the following:
- IP ranges to allow incoming traffic from specific IP addresses. Specify the range of IP addresses in the Source IP ranges field.
- Subnets to allow incoming traffic from a particular subnetwork. Specify the subnetwork name in the following Subnets field. You can use this option to allow access between the VMs in a 3-tier or scaleout configuration.
- In the Protocols and ports section, select Specified protocols and
ports and enter
Click Create to create your firewall rule.
Create a firewall rule by using the following command:
$gcloud compute firewall-rules create firewall-name --direction=INGRESS --priority=1000 \ --network=network-name --action=ALLOW --rules=protocol:port \ --source-ranges ip-range --target-tags=network-tags
Creating a high-availability Linux cluster with SAP HANA installed
The following instructions use the Cloud Deployment Manager to create a SLES Linux cluster with two SAP HANA systems: a primary single-host SAP HANA system on one VM instance and a standby SAP HANA system on another VM instance in the same Compute Engine region. The SAP HANA systems use synchronous system replication and the standby system preloads the replicated data.
You define configuration options for the SAP HANA high-availability cluster in a Deployment Manager configuration file template.
The following instructions use the Cloud Shell, but are generally applicable to the Cloud SDK.
Confirm that your current quotas for resources such as persistent disks and CPUs are sufficient for the SAP HANA systems you are about to install. If your quotas are insufficient, deployment fails. For the SAP HANA quota requirements, see Pricing and quota considerations for SAP HANA.
Open the Cloud Shell or, if you installed the Cloud SDK on your local workstation, open a terminal.
In the Cloud Shell, from the subnetwork range that you are using for your HA cluster, reserve an IP address to use as your VIP:
$gcloud compute addresses create vip-name \ --region cluster-region --subnet cluster-subnet \ --addresses vip-address
template.yamlconfiguration file template for the SAP HANA high-availability cluster to your working directory by entering the following command in the Cloud Shell or Cloud SDK:
Optionally, rename the
template.yamlfile to identify the configuration it defines.
template.yamlfile in the Cloud Shell code editor or, if you are using the Cloud SDK, the text editor of your choice.
To open the Cloud Shell code editor, click the pencil icon in the upper right corner of the Cloud Shell terminal window.
template.yamlfile, update the property values by replacing the brackets and their contents with the values for your installation. The properties are described in the following table.
To create the VM instances without installing SAP HANA, delete or comment out all of the lines that begin with
Property Data type Description type String
Specifies the location, type, and version of the Deployment Manager template to use during deployment.
The YAML file includes two
typespecifications, one of which is commented out. The
typespecification that is active by default specifies the template version as
typespecification that is commented out specifies a specific template version with a timestamp.
If you need all of your deployments to use the same template version, use the
typespecification that includes the timestamp.
primaryInstanceName String The name of the VM instance for the primary SAP HANA system. Specify the name in lowercase letters, numbers, or hyphens. If any other characters are used, such as "" (underscore) or a capital letter, deployment fails. secondaryInstanceName String The name of the VM instance for the secondary SAP HANA system. Specify the name in lowercase letters, numbers, or hyphens. If any other characters are used, such as "" (underscore) or a capital letter, deployment fails. primaryZone String The zone in which the primary SAP HANA system is deployed. The primary and secondary zones must be in the same region. secondaryZone String The zone in which the secondary SAP HANA system will be deployed. The primary and secondary zones must be in the same region. instanceType String The type of Compute Engine virtual machine that you need to run SAP HANA on. If you need a custom VM type, specify a predefined VM type with a number of vCPUs that is closest to the number you need while still being larger. After deployment is complete, modify the number of vCPUs and the amount of memory. subnetwork String The name of the subnetwork you created in a previous step. If you are deploying to a shared VPC, specify this value as
[SHAREDVPC_PROJECT]/[SUBNETWORK]. For example,
linuxImage String The name of the Linux operating- system image or image family that you are using with SAP HANA. To specify an image family, add the prefix
family/to the family name. For example,
family/sles-12-sp3-sap. To specify a specific image, specify only the image name. For the list of available image families, see the Images page in the Cloud console.
linuxImageProject String The Google Cloud project that contains the image you are going to use. This project might be your own project or a Google Cloud image project. For SLES, specify
suse-sap-cloud. For a list of GCP image projects, see the Images page in the Compute Engine documentation.
sap_hana_deployment_bucket String The name of the GCP storage bucket in your project that contains the SAP HANA installation files that you uploaded in a previous step. sap_hana_sid String The SAP HANA system ID. The ID must consist of three alphanumeric characters and begin with a letter. All letters must be uppercase. sap_hana_instance_number Integer The instance number, 0 to 99, of the SAP HANA system. The default is 0. sap_hana_sidadm_password String A temporary password for the operating system administrator. Passwords must be at least eight characters and include at least one uppercase letter, one lowercase letter, and one number. sap_hana_system_password String A temporary password for the database superuser. Passwords must be at least 8 characters and include at least one uppercase letter, one lowercase letter, and one number. sap_hana_scaleout_nodes Integer The number of additional SAP HANA worker hosts that you need. Specify
0, because scaleout hosts are not supported in high availability configurations currently.
sap_vip String The IP address that you reserved earlier for the VIP. This IP address is always assigned to the active SAP HANA instance. The IP address must be within the range of IP addresses that are assigned to your subnetwork.
The following example shows a completed configuration file, which directs the Deployment Manager to deploy a high-availability cluster with the primary SAP HANA system installed in the
us-central1-czone and the secondary SAP HANA system installed in the
us-central1-fzone. Both systems will be installed on
n1-highmem-32VMs that are running the SLES 12 SP3 operating system.
resources: - name: sap_hana_ha type: https://storage.googleapis.com/cloudsapdeploy/deploymentmanager/latest/dm-templates/sap_hana_ha/sap_hana_ha.py # # By default, this configuration file uses the latest release of the deployment # scripts for SAP on Google Cloud. To fix your deployments to a specific release # of the scripts, comment out the type property above and uncomment the type property below. # # type: https://storage.googleapis.com/cloudsapdeploy/deploymentmanager/yyyymmddhhmm/dm-templates/sap_hana_ha/sap_hana_ha.py # properties: primaryInstanceName: example-ha-vm1 secondaryInstanceName: example-ha-vm2 primaryZone: us-central1-c secondaryZone: us-central1-f instanceType: n1-highmem-32 subnetwork: example-ha-subnetwork linuxImage: family/sles-12-sp3-sap linuxImageProject: suse-sap-cloud sap_hana_deployment_bucket: hana2sp3rev30 sap_hana_sid: HA1 sap_hana_instance_number: 00 sap_hana_sidadm_password: TempPa55word sap_hana_system_password: TempPa55word sap_hana_scaleout_nodes: 0 sap_vip: 10.1.0.24
Create the instances:
$gcloud deployment-manager deployments create deployment-name --config template-name.yaml
The above command invokes the Deployment Manager, which deploys the VMs, downloads the SAP HANA software from your storage bucket, and installs SAP HANA, all according to the specifications in your template.yaml file. The process usually takes less than 30 minutes to complete. To check the progress of your deployment, follow the steps in the next section.
Verifying the deployment of your HANA HA system
Verifying an SAP HANA HA cluster involves several different procedures:
- Checking the Logging logs
- Checking the configuration of the VM and the SAP HANA installation
- Checking the SAP HANA system using SAP HANA Studio
- Performing a failover test
Checking the Logging logs
Open Logging to check for errors and monitor the progress of the installation.
On the Resources tab, select Global as your logging resource.
"INSTANCE DEPLOYMENT COMPLETE"is displayed, Deployment Manager processing is complete and you can proceed to the next step.
If you see a quota error:
- On the IAM & admin Quotas page, increase any of your quotas that do not meet the SAP HANA requirements that are listed in the SAP HANA Planning Guide.
- On the Deployment Manager Deployments page, delete the deployment to clean up the VMs and persistent disks from the failed installation.
- Rerun the Deployment Manager.
Checking the configuration of the VM and the SAP HANA installation
After the SAP HANA system deploys without errors, connect to each VM by using SSH. From the Compute Engine VM instances page, you can click the SSH button for each VM instance, or you can use your preferred SSH method.
Change to the root user.
sudo su -
At the command prompt, enter
df -h. Ensure that you see output that includes the
/hanadirectories, such as
example-ha-vm1:~ # df -h Filesystem Size Used Avail Use% Mounted on devtmpfs 103G 8.0K 103G 1% /dev tmpfs 154G 54M 154G 1% /dev/shm tmpfs 103G 35M 103G 1% /run tmpfs 103G 0 103G 0% /sys/fs/cgroup /dev/sda1 30G 5.1G 24G 18% / /dev/mapper/vg_hana-shared 204G 49G 156G 24% /hana/shared /dev/mapper/vg_hana-sap 32G 239M 32G 1% /usr/sap /dev/mapper/vg_hana-data 496G 7.0G 489G 2% /hana/data /dev/mapper/vg_hana-log 102G 4.2G 98G 5% /hana/log /dev/mapper/vg_hanabackup-backup 416G 5.5G 411G 2% /hanabackup tmpfs 21G 0 21G 0% /run/user/900 tmpfs 21G 0 21G 0% /run/user/489 tmpfs 21G 0 21G 0% /run/user/1003
Check the status of the new cluster:
You should see results similar to the following the example, in which both VM instances are started and
example-ha-vm1is the active primary instance and
example-ha-vm2is the secondary instance:
example-ha-vm1:~ # crm status Stack: corosync Current DC: example-ha-vm1 (version 1.1.16-6.26.1-77ea74d) - partition with quorum Last updated: Fri Jul 9 19:47:21 2021 Last change: Fri Jul 9 19:47:19 2021 by root via crm_attribute on example-ha-vm1 2 nodes configured 8 resources configured Online: [ example-ha-vm1 example-ha-vm2 ] Full list of resources: STONITH-example-ha-vm1 (stonith:external/gcpstonith): Started example-ha-vm2 STONITH-example-ha-vm2 (stonith:external/gcpstonith): Started example-ha-vm1 Resource Group: g-primary rsc_vip_int-primary (ocf::heartbeat:IPaddr2): Started example-ha-vm1 rsc_vip_gcp-primary (ocf::gcp:alias): Started example-ha-vm1 Clone Set: cln_SAPHanaTopology_HA1_HDB00 [rsc_SAPHanaTopology_HA1_HDB00] Started: [ example-ha-vm1 example-ha-vm2 ] Master/Slave Set: msl_SAPHana_HA1_HDB00 [rsc_SAPHana_HA1_HDB00] Masters: [ example-ha-vm1 ] Slaves: [ example-ha-vm2 ]
Change to the SAP admin user by replacing [SID] in the following command with the [SID] value that you specified in the configuration file template.
su - [SID]adm
Ensure that the SAP HANA services, such as
hdbindexserver, and others, are running on the instance by entering the following command:
Checking the SAP HANA system using SAP HANA Studio
Connect to the HANA system by using SAP HANA Studio. When defining the connection, specify the following values:
- On the Specify System panel, specify the floating IP address as the Host Name.
- On the Connection Properties panel, for database user authentication, specify the database superuser name and the password that you specified for the sap_hana_system_password property in the template.yaml file.
For information from SAP about installing SAP HANA Studio, see SAP HANA Studio Installation and Update Guide.
After SAP HANA Studio is connected to your HANA HA system, display the system overview by double-clicking the system name in the navigation pane on the left side of the window.
Under General Information on the Overview tab, confirm that:
- The Operational Status shows "All services started".
- The System Replication Status shows "All services are active and in sync".
Confirm the replication mode by clicking the System Replication Status link under General Information. Synchronous replication is indicated by
SYNCMEMin the REPLICATION_MODE column on the System Replication tab.
If any of the validation steps show that the installation failed:
- Resolve the errors.
- Delete the deployment from the Deployments page.
- Recreate the instances, as described in the last step of the previous section.
Performing a failover test
To perform a failover test:
Connect to the primary VM by using SSH. You can connect from the Compute Engine VM instances page by clicking the SSH button for each VM instance, or you can use your preferred SSH method.
At the command prompt, enter the following command:
sudo ip link set eth0 down
ip link set eth0 downcommand triggers a failover by severing communications with the primary host.
Reconnect to either host using SSH and change to the root user.
crm statusto confirm that the primary host is now active on the VM that used to contain the secondary host. Automatic restart is enabled in the cluster, so the stopped host will restart and assume the role of secondary host, as shown in the following screenshot.
example-ha-vm2:~ # crm status Stack: corosync Current DC: example-ha-vm2 (version 1.1.16-6.26.1-77ea74d) - partition with quorum Last updated: Fri Jul 9 20:21:33 2021 Last change: Fri Jul 9 20:20:56 2021 by root via crm_attribute on example-ha-vm2 2 nodes configured 8 resources configured Online: [ example-ha-vm1 example-ha-vm2 ] Full list of resources: STONITH-example-ha-vm1 (stonith:external/gcpstonith): Started example-ha-vm2 STONITH-example-ha-vm2 (stonith:external/gcpstonith): Started example-ha-vm1 Resource Group: g-primary rsc_vip_int-primary (ocf::heartbeat:IPaddr2): Started example-ha-vm2 rsc_vip_gcp-primary (ocf::gcp:alias): Started example-ha-vm2 Clone Set: cln_SAPHanaTopology_HA1_HDB00 [rsc_SAPHanaTopology_HA1_HDB00] Started: [ example-ha-vm1 example-ha-vm2 ] Master/Slave Set: msl_SAPHana_HA1_HDB00 [rsc_SAPHana_HA1_HDB00] Masters: [ example-ha-vm2 ] Slaves: [ example-ha-vm1 ]
In SAP HANA Studio, confirm that you are still connected to the system by double-clicking the system entry in the navigation pane to refresh the system information.
Click the System Replication Status link to confirm that the primary and secondary hosts have switched hosts and are active.
Setting up the Google monitoring agent for SAP HANA
Optionally, you can set up the Google monitoring agent for SAP HANA, which collects metrics from SAP HANA and sends them to Monitoring. Monitoring lets you create dashboards for your metrics, set up custom alerts based on metric thresholds, and more.
To monitor an HA cluster, you can install the monitoring agent on either a VM instance outside of the cluster or on each VM instance in the cluster.
If you install the monitoring agent on a VM instance outside of the HA cluster, you specify the floating IP address of the cluster as the IP address of the host instance to monitor. If you install the monitoring agent on each VM in the cluster, when configuring each monitoring agent, you specify the local IP address of the VM instance that is hosting the monitoring agent.
For more information on setting up and configuring the Google monitoring agent for SAP HANA, see the SAP HANA Monitoring Agent User Guide.
Connecting to SAP HANA
Note that because these instructions don't use an external IP address for SAP HANA, you can only connect to the SAP HANA instances through the bastion instance using SSH or through the Windows server through SAP HANA Studio.
To connect to SAP HANA through the bastion instance, connect to the bastion host, and then to the SAP HANA instance(s) by using an SSH client of your choice.
To connect to the SAP HANA database through SAP HANA Studio, use a remote desktop client to connect to the Windows Server instance. After connection, manually install SAP HANA Studio and access your SAP HANA database.
Performing post-deployment tasks
Before using your SAP HANA instance, we recommend that you perform the following post-deployment steps. For more information, see SAP HANA Installation and Update Guide.
Change the temporary passwords for the SAP HANA system administrator and database superuser. For example:
sudo passwd [SID]adm
Update the SAP HANA software with the latest patches.
Install any additional components such as Application Function Libraries (AFL) or Smart Data Access (SDA).
Configure and backup your new SAP HANA database. For more information, see the SAP HANA operations guide.
- If you need to change the VIP implementation in your HA cluster from an alias IP implementation to an internal load balancer implementation, follow the instructions in Migrating a VIP in an SAP HANA HA cluster to an internal load balancer.
- For more information about VM administration of and monitoring, see the SAP HANA Operations Guide.