This tutorial walks you through the process of migrating a MySQL database to
Google Cloud by using native
MySQL replication, HAProxy
instances, and Compute Engine. MySQL is a
popular, multi-purpose relational
database management system (RDBMS).
This tutorial is useful if you are a sysadmin, developer, engineer, database admin, or devops engineer who wants to migrate your data on your existing MySQL cluster to MySQL on Compute Engine. You might want to manage your own MySQL instance instead of using the managed service, due to cross-region instances, advanced usage of parameters, and specific performance needs. This solution doesn't address the migration of MySQL to Cloud SQL.
The tutorial assumes that you are familiar with the following:
- Linux
- Ubuntu-server 16.04
- MySQL 5.7
- HAProxy
- Compute Engine
Architecture
You use a Cloud Deployment Manager template from this tutorial's GitHub
repository to create an environment where a MySQL cluster is in the us-east1
region, consisting of a primary (master
) and a replica (slave
), as well as a
MySQL client. The scripts detailed in this document populate the database with a
source_db
schema and a source_table
table with 5000 rows of sample data.
The following architectural diagram illustrates your environment when you begin. It consists of a MySQL cluster and a client instance.
First, you replicate the source replica instance into the target primary
instance running on Compute Engine and create an HAProxy
instance to
forward the traffic. In the following diagram, the HAProxy load balancer points
to the source primary, and replication is configured between the source replica
and the target primary instance:
After replication is configured and the deployments are in sync, you point the HAProxy load balancer to the target primary Compute Engine instance, as illustrated in the following diagram:
Objectives
- Create a source MySQL cluster and a MySQL client instance on Compute Engine using Deployment Manager.
- Set up a one-node target MySQL deployment on Compute Engine.
- Replicate data from the source MySQL cluster to the target MySQL instance.
- Create an
HAProxy
instance and configure it to point to the source MySQL cluster. - Point the MySQL client to the
HAProxy
instance. - Point the
HAProxy
instance to the target MySQL node. - Stop the data replication from the source deployment to the target deployment.
Costs
This tutorial uses the following billable components of Google Cloud:
- Compute Engine
- Cloud Storage
To generate a cost estimate based on your projected usage,
use the pricing calculator.
Before you begin
- Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.
-
In the Google Cloud console, on the project selector page, select or create a Google Cloud project.
-
Make sure that billing is enabled for your Cloud project. Learn how to check if billing is enabled on a project.
-
In the Google Cloud console, on the project selector page, select or create a Google Cloud project.
-
Make sure that billing is enabled for your Cloud project. Learn how to check if billing is enabled on a project.
-
Enable the Compute Engine and Cloud Deployment Manager APIs.
When you finish this tutorial, you can avoid continued billing by deleting the resources you created. See Cleaning up for more detail.
Setting up your environment
In this tutorial, you use Cloud Shell to enter commands. Cloud Shell gives you access to the command line in the Google Cloud console and includes Google Cloud CLI and other tools that you need to develop in Google Cloud. Cloud Shell appears as a window at the bottom of the Google Cloud console. It can take several minutes to initialize, but the window appears immediately.
As a first step, you use Cloud Shell to create a Cloud Storage bucket.
Open Cloud Shell.
Set an environment variable for the Cloud Storage bucket name.
GCS_BUCKET_NAME=${USER}-mysql-$(date +%s) echo $GCS_BUCKET_NAME
Create the Cloud Storage bucket.
gsutil mb gs://${GCS_BUCKET_NAME}/
Retrieve the scripts to set up the environment from the GitHub repository.
git clone https://github.com/GoogleCloudPlatform/solutions-compute-mysql-migration-haproxy.git mysql-migration
Run the initialization script to create a MySQL cluster of primary and replica instances. This script also creates a MySQL client instance.
cd mysql-migration ./run.sh ${DEVSHELL_PROJECT_ID} ${GCS_BUCKET_NAME}
Preparing the source MySQL for replication
To migrate the data to another MySQL instance, you set up the
source-mysql-replica
instance to replicate the data to other instances.
In the Google Cloud console, go to the VM instances page.
In the
source-mysql-replica
instance row, click ssh.Specify a user for replication purposes. For the purposes of this tutorial, the password for this user is
solution-admin
.mysql -u root -psolution-admin -e "GRANT REPLICATION SLAVE ON *.* TO 'sourcereplicator'@'%' IDENTIFIED BY 'solution-admin';"
Enable replica logging.
sudo bash -c 'echo log_slave_updates = 1 >>/etc/mysql/mysql.conf.d/mysqld.cnf'
Restart MySQL.
sudo service mysql restart
When you restart MySQL, any clients connected to the replica instance are disconnected. It might take time for the replica to synchronize with the primary, depending on the volume of writes in the primary while the replica is restarting.
Setting up the service account for target MySQL instances
A service account is associated with a set of roles and permissions. You create a service account for your MySQL instance that has the minimum privileges required for this tutorial.
Open Cloud Shell.
Create the service account.
gcloud iam service-accounts create mysql-instance \ --display-name "mysql-instance"
Add the
storage.objectAdmin
role to themysql-instance
service account. This role is required in order to view and download files from the Cloud Storage bucket.gcloud projects add-iam-policy-binding ${DEVSHELL_PROJECT_ID} \ --member=serviceAccount:mysql-instance@${DEVSHELL_PROJECT_ID}.iam.gserviceaccount.com \ --role=roles/storage.objectAdmin
Setting up the target MySQL instance on Compute Engine
In this section, you create a Compute Engine instance and install MySQL on it.
In Cloud Shell, create the target MySQL instance.
gcloud compute instances create target-mysql-primary \ --image-family=ubuntu-1604-lts --image-project=ubuntu-os-cloud \ --tags=mysql57 --zone=us-central1-c \ --service-account=mysql-instance@${DEVSHELL_PROJECT_ID}.iam.gserviceaccount.com \ --scopes=https://www.googleapis.com/auth/devstorage.read_write
In the Google Cloud console, go to the VM instances page.
Click Refresh.
In the Google Cloud console, use
ssh
to connect to thetarget-mysql-primary
instance.In the terminal window of the instance, install MySQL.
sudo apt-get update sudo apt-get -y install mysql-server-5.7
When prompted, enter
solution-admin
for the root user password.In Cloud Shell, create firewall rules to allow communication between the source MySQL instances and the target instance.
gcloud compute firewall-rules create mysql --allow=tcp:3306 \ --source-tags source-mysql --target-tags target-mysql
Setting up secured root access between the source replica and target primary instances
You set up authentication by creating private and public keys on the
target-mysql-primary
instance and copying the public key to the
source-mysql-replica
instance. This authentication enables you to quickly copy
files later from the source-mysql-replica
instance to the
target-mysql-primary
instance by using the rsync
command.
In the Google Cloud console, use
ssh
to connect to thetarget-mysql-primary
instance.Create a private key.
sudo ssh-keygen
When prompted, press
Enter
to accept all default settings.Copy the public key to your Cloud Storage bucket.
sudo bash -c "gsutil cp /root/.ssh/id_rsa.pub gs://[GCS_BUCKET_NAME]/"
Where:
[GCS_BUCKET_NAME]
represents the bucket you created at the beginning of this tutorial.
In the Google Cloud console, use
ssh
to connect to thesource-mysql-replica
instance.In the terminal window of the instance, copy the public key from the Cloud Storage bucket.
sudo bash -c "gsutil cp \ gs://[GCS_BUCKET_NAME]/id_rsa.pub /root/.ssh/target-mysql-primary.pub"
Where:
[GCS_BUCKET_NAME]
represents the bucket you created at the beginning of this tutorial.
Add the public key to the
authorized_keys
file.sudo bash -c "cat /root/.ssh/target-mysql-primary.pub >> \ /root/.ssh/authorized_keys"
In the Google Cloud console, use
ssh
to connect to thetarget-mysql-primary
instance.In the
target-mysql-primary
instance, test your connection.sudo ssh source-mysql-replica
When prompted, enter
yes
to accept the remote server's public key.
Syncing the MySQL deployments
You use the rsync
command to copy the MySQL data files from the
source-mysql-replica
instance to the target-mysql-primary
instance. To
minimize downtime, you run this command twice: once while the source replica is
online to copy the majority of the data, and then a second time while the source
replica is offline. Copying the files while the source replica is offline
ensures that all the data in the open files are copied correctly.
Copy MySQL data files
In this section, you copy the MySQL data files from the source-mysql-replica
instance to the target-mysql-primary
instance.
In the Google Cloud console, use
ssh
to connect to thetarget-mysql-primary
instance.In the terminal window of the instance, stop MySQL.
sudo service mysql stop
Delete the contents of
/var/lib/mysql
.sudo bash -c "rm -rf /var/lib/mysql/*"
Copy the database files from the source.
sudo bash -c "rsync -av source-mysql-replica:/var/lib/mysql/ /var/lib/mysql"
In this tutorial, this process takes 5-10 seconds. However, for a real workload, the time to complete this process depends on the size of the database you are migrating and the performance of your network.
In the Google Cloud console, use
ssh
to connect to thesource-mysql-replica
instance.Pause the replication from the
source-mysql-primary
instance.mysql -uroot -psolution-admin -e 'show master status; stop slave;'
The output appears in the following format.
+------------------+----------+--------------+------------------+-------------------+ | File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set | +------------------+----------+--------------+------------------+-------------------+ | mysql-bin.000002 | 154 | source_db | | | +------------------+----------+--------------+------------------+-------------------+
Make a note of the
log_bin
file and position. Later in the tutorial, themysql-bin.000002
file name is represented asPRIMARY_LOG_FILE
and the154
file position is represented asPRIMARY_LOG_POS
, because your values might be different.On the
target-mysql-primary
instance, run the following command to ensure that thetarget-mysql-primary
instance is consistent with thesource-mysql-replica
instance, because you didn't stop the writes during the first copy operation.sudo bash -c "rsync -av source-mysql-replica:/var/lib/mysql/ /var/lib/mysql"
On the
source-mysql-replica
instance, resume the replication.sudo mysql -uroot -psolution-admin -e 'start slave;'
Configure the target-mysql-primary instance for replication
In the Google Cloud console, use
ssh
to connect to thetarget-mysql-primary
instance.Remove the file containing the default MySQL instance ID.
sudo rm /var/lib/mysql/auto.cnf
Update the MySQL configuration to replicate the
source_db
database from thesource-mysql-replica
instance.sudo sed -i "s|#server-id.*|server-id = 4|" \ /etc/mysql/mysql.conf.d/mysqld.cnf sudo sed -i "s|#log_bin|log_bin|" /etc/mysql/mysql.conf.d/mysqld.cnf sudo sed -i "s|#binlog_do_db.*|binlog_do_db = source_db|" \ /etc/mysql/mysql.conf.d/mysqld.cnf
Enable MySQL to accept connections from other hosts on its network.
LOCAL_IP=$(curl http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/ip \ -H "Metadata-Flavor: Google") sudo sed -i "s|bind-address.*|bind-address = $LOCAL_IP|" \ /etc/mysql/mysql.conf.d/mysqld.cnf
Start MySQL.
sudo service mysql start
Log in to the MySQL console.
mysql -u root -psolution-admin
Reset the primary instance.
reset slave;
Configure the replication process.
CHANGE MASTER TO MASTER_HOST='source-mysql-replica', \ MASTER_USER='sourcereplicator', MASTER_PASSWORD='solution-admin', \ MASTER_LOG_FILE='[[PRIMARY_LOG_FILE]]', MASTER_LOG_POS=[[PRIMARY_LOG_POS]];
Where:
[PRIMARY_LOG_FILE]
representsmysql-bin.000002
from an earlier step.[PRIMARY_LOG_FILE]
represents154
from an earlier step.
Start the replication.
start slave;
Verify that the
source_db
table exists on the server and contains the data from the source deployment.SELECT * FROM source_db.source_table;
The table appears in the following format:
+----+---------------------+------------+ | id | timestamp | event_data | +----+---------------------+------------+ | 1 | 2018-09-06 13:57:17 | 8511.85 | | 2 | 2018-09-06 13:57:17 | 2658.33 | | 3 | 2018-09-06 13:57:17 | 2756.08 | | 4 | 2018-09-06 13:57:17 | 5805.42 | | 5 | 2018-09-06 13:57:17 | 5758.86 |
Check the status of the replica instance.
show slave status \G
The output is similar to the following.
*************************** 1. row *************************** Slave_IO_State: Waiting for master to send event Master_Host: source-mysql-replica ... Slave_IO_Running: Yes Slave_SQL_Running: Yes ... 1 row in set (0.00 sec)
Creating the HAProxy instance
By using the HAProxy
instance, you create one access point that you can
point to the source deployment. Later you can point that access point to the
target deployment,
without reconfiguring the clients during the migration.
In Cloud Shell, create the
HAProxy
instance and assign a fixed IP address to the instance.gcloud compute instances create haproxy --image-family=ubuntu-1604-lts \ --image-project=ubuntu-os-cloud --tags=haproxy,http-server \ --zone=us-central1-c --private-network-ip=10.128.0.100
In the Google Cloud console, go to the VM instances page.
Click Refresh.
On the row for the
HAProxy
instance, click ssh to connect to the instance.On the
HAProxy
instance, install HAProxy 1.6.3.sudo apt-get update sudo apt-get install -y haproxy=1.6.3-1
To be able to test connectivity to the MySQL instances later, install the MySQL client.
sudo apt-get install -y mysql-client
Enabling the HAProxy instance to communicate with the primary nodes of both MySQL deployments
When you finish setting up the MySQL replication, you are ready to start the migration.
Enable firewall rules
In Cloud Shell, create firewall rules to enable communication between the
HAProxy
instance and both MySQL deployments.gcloud compute firewall-rules create haproxy-mysql --allow=tcp:3306 \ --source-tags haproxy --target-tags target-mysql,source-mysql
Create firewall rules to allow
HAProxy
instance admin access.gcloud compute firewall-rules create haproxy-admin --allow=tcp:80 \ --source-ranges=0.0.0.0/0 --target-tags haproxy
Set up HAProxy authentication for the target instance
In this section, you set up HAProxy
authentication for the
target-mysql-primary
instance.
In the Google Cloud console, use
ssh
to connect to thetarget-mysql-primary
instance.Create a user called
haproxy_check
. Authorize the user to log in from theHAProxy
instance. TheHAProxy
instance uses this user to check the status of a server.mysql -uroot -psolution-admin -e "INSERT INTO mysql.user \ (Host,User,ssl_cipher,x509_issuer,x509_subject) values \ ('10.128.0.100','haproxy_check','','',''); FLUSH PRIVILEGES";
Create a user called
haproxy_root
. Authorize the user to log in from theHAProxy
instance.PROJECT_ID=`curl \ http://metadata.google.internal/computeMetadata/v1/project/project-id -H "Metadata-Flavor: Google"` HA_PROXY_FQDN=haproxy.c.$PROJECT_ID.internal mysql -uroot -psolution-admin -e "GRANT ALL PRIVILEGES ON *.* TO \ 'haproxy_root'@'"$HA_PROXY_FQDN"' IDENTIFIED BY 'solution-admin' WITH \ GRANT OPTION; FLUSH PRIVILEGES;"
Set up HAProxy authentication for the source primary instance
In this section, you set up HAProxy
authentication for the
source-mysql-primary
instance.
In the Google Cloud console, use
ssh
to connect to thesource-mysql-primary
instance.Create a user called
haproxy_check
and give them permission to log in from theHAProxy
instance.mysql -uroot -psolution-admin -e "INSERT INTO mysql.user \ (Host,User,ssl_cipher,x509_issuer,x509_subject) values \ ('10.128.0.100','haproxy_check','','',''); FLUSH PRIVILEGES;"
Create a user called
HAProxy_root
and give them permission to log in from theHAProxy
instance.PROJECT_ID=`curl \ http://metadata.google.internal/computeMetadata/v1/project/project-id -H \ "Metadata-Flavor: Google"` HA_PROXY_FQDN=haproxy.c.$PROJECT_ID.internal mysql -uroot -psolution-admin -e "GRANT ALL PRIVILEGES ON *.* TO \ 'haproxy_root'@'"$HA_PROXY_FQDN"' IDENTIFIED BY 'solution-admin' WITH GRANT OPTION; FLUSH PRIVILEGES;"
This
haproxy_root
user is needed in order to access MySQL from theHAProxy
instance.
Test connectivity between HAProxy and the MySQL primary instances
In the Google Cloud console, use
ssh
to log in to the HAProxy machine.Test connectivity with the
source-mysql-primary
instance.mysql -h source-mysql-primary -u haproxy_root -psolution-admin -e \ "SHOW DATABASES"
Test connectivity with the
target-mysql-primary
instance.mysql -h target-mysql-primary -u haproxy_root -psolution-admin -e \ "SHOW DATABASES"
Configuring HAProxy
Now you create the HAProxy
configuration file and point it to the
source-mysql-primary
instance.
In the Google Cloud console, use
ssh
to log in to theHAProxy
instance.Back up the existing
HAProxy
configuration file on theHAProxy
instance.sudo cp /etc/haproxy/haproxy.cfg /etc/haproxy/haproxy.cfg.bkp
Configure the
HAProxy
instance and create a password to be used later in the tutorial.sudo bash -c "cat <<EOF > /etc/haproxy/haproxy.cfg ######### HAProxy Config file ######### global log /dev/log local0 log /dev/log local1 notice chroot /var/lib/haproxy stats socket /run/haproxy/admin.sock mode 660 level admin stats timeout 30s user haproxy group haproxy daemon defaults log global timeout connect 3000 timeout client 5000 timeout server 5000 listen mysql-cluster bind 127.0.0.1:3306,$(curl \ http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/ip \ -H 'Metadata-Flavor: Google'):3306 mode tcp option mysql-check user haproxy_check balance roundrobin # Server number 1 server source-primary source-mysql-primary:3306 check # Server number 2 # server target-primary target-mysql-primary:3306 check listen stats bind 0.0.0.0:80 mode http stats enable stats uri /haproxy stats realm Strictly\ Private stats auth mysqlproxy:MySQLProxy12! EOF"
Reload the
HAProxy
service.sudo service haproxy reload
Testing your deployment
From the
HAProxy
instance use the localhost address as the host, to connect to thesource-mysql-primary
instance.mysql -h 127.0.0.1 -u haproxy_root -psolution-admin -e "SHOW DATABASES"
The output contains the names of the databases.
+--------------------+ | Database | +--------------------+ | information_schema | | mysql | | performance_schema | | source_db | | sys | +--------------------+
In the Google Cloud console, use
ssh
to connect to themysql-client
instance.Use the
HAProxy
instance to test connectivity to thesource-mysql-primary
instance.mysql -h haproxy -u haproxy_root -psolution-admin -e "SHOW DATABASES"
In Cloud Shell, find the public IP address of the
HAProxy
instance.gcloud compute instances describe haproxy \ --format='value(networkInterfaces[0].accessConfigs[0].natIP)' \ --zone=us-central1-c
Copy the external IP address of the
HAProxy
instance.To check the connectivity status between the
HAProxy
instance and thesource-mysql-master
instance, go to the following URL and verify that the source-primary row is green.http://[PUBLIC_IP_OF_HAPROXY]/haproxy
Where:
[PUBLIC_IP_OF_HAPROXY]
represents the public IP address of yourHAProxy
instance.
In the Username field, enter
mysqlproxy
.In the Password field, enter
MySQLProxy12!
.
Switching the MySQL server
In this stage, you perform the actual migration. You make sure that all instances are in sync, point the HAProxy service to the target deployment, and test your environment.
Stop the HAProxy service
Connect to the
HAProxy
instance.To ensure that the target matches the source, stop any connections to the
source-mysql-primary
instance.sudo service haproxy stop
Lock tables in the primary instance
Connect to the
source-mysql-primary
instance.Lock the tables for writes.
mysql -uroot -psolution-admin -e "FLUSH TABLES WITH READ LOCK"
Verify that the source instances are in sync
In this section, you verify that the source-mysql-primary
and the
source-mysql-replica
instances are in sync.
In the Google Cloud console, use
ssh
to log in to thesource-mysql-replica
instance.Check the status of the replica.
mysql -u root -psolution-admin -e "show slave status \G" | grep \ Seconds_Behind_Master
The instances are in sync if the value for the
Seconds_Behing_Master
is0
.
Verify that the source and target instances are in sync
In this section, you verify that the source-mysql-replica
and the
target-mysql-primary
instances are in sync.
In the Google Cloud console, use
ssh
to log in to thetarget-mysql-primary
instance.Check the status of the replica.
mysql -u root -psolution-admin -e "show slave status \G" | grep Seconds_Behind_Master
The instances are in sync if the value for the
Seconds_Behind_Master
is0
.Stop the replication from the
source-mysql-replica
instancemysql -u root -psolution-admin -e "stop slave;reset master;"
Point the HAProxy configuration to the target-mysql-primary instance
In the Google Cloud console, use
ssh
to log in to theHAProxy
instance.Change the
HAProxy
configuration to point to thetarget-mysql-primary
instance.sudo sed -i 's|server source-primary|# server source-primary|g' \ /etc/haproxy/haproxy.cfg sudo sed -i 's|# server target-primary|server target-primary|g' \ /etc/haproxy/haproxy.cfg
Reload the service.
sudo service haproxy restart
To check the connectivity status between the
HAProxy
instance to thetarget-mysql-primary
instance, go to the following URL and verify that the target-primary row is green.http://[PUBLIC_IP_OF_HAPROXY]/haproxy
Where:
[PUBLIC_IP_OF_HAPROXY]
represents the public IP address of yourHAProxy
instance.
In the Username field, enter
mysqlproxy
.In the Password field, enter
MySQLProxy12!
.In the Google Cloud console, use
ssh
to log in to themysql-client
instance.Test the new configuration.
mysql -h haproxy -u haproxy_root -psolution-admin -e "SHOW DATABASES"
The output contains the names of the databases.
+--------------------+ | Database | +--------------------+ | information_schema | | mysql | | performance_schema | | source_db | | sys | +--------------------+
Considerations for migrating a production cluster
In this tutorial, you migrated a MySQL cluster into a different MySQL instance. The source cluster, even though it is deployed on Google Cloud, is meant to simulate a cluster that is on-premises or on a different cloud platform. Here are some things to consider when migrating a production workload:
- Setting up connectivity: Whether you are migrating from an on-premises deployment or from another public cloud, you need to set up connectivity channels between your environment and Google Cloud. Consider using a VPN or Cloud Interconnect.
- Network performance: What is the latency between your source environment and your target environment? What is your available bandwidth? Your factors can have significant effects on your sync speed and affect your connection between your clients and the HAProxy cluster.
- Database parameter fine-tuning: This tutorial only addresses some replication parameters. Your database admins might have configured the source MySQL cluster with different custom parameters to modify its performance and configuration based on the needs of your environment and app. For example, you might need to adjust the target cluster configurations.
- The types of queries you are running against your MySQL cluster: Are
you running long queries? Is the number of queries per second high or low?
Your answers might affect how you configure your
HAProxy
instance. For example, you might need to adjust timeouts. - Target deployment: In this tutorial, you haven't set up a replica instance for the target deployment. For a production environment, you set up a cluster for the target deployment.
Haproxy
instances and deployments: You used a singleHAProxy
instance for this tutorial. For a production workload migration, you can use a redundant HAProxy deployment.- Zone selection: This tutorial uses the
us-central1-c
zone, but you can use any zone for your production workloads. For more information on how to select a region, see Best Practices for Compute Engine Region Selection. - Disk size: Make sure the size of the disk matches the disk size of the source MySQL instances.
- Permissions and access: You used the root user in this tutorial. In a
production migration, you might need to use a different user if you can't
run the
sudo
orroot
commands.
Clean up
To avoid incurring charges to your Google Cloud account for the resources used in this tutorial, either delete the project that contains the resources, or keep the project and delete the individual resources.
Delete the project
- In the Google Cloud console, go to the Manage resources page.
- In the project list, select the project that you want to delete, and then click Delete.
- In the dialog, type the project ID, and then click Shut down to delete the project.
What's next
- Learn how to set up redundant HAProxy with real-world examples.
- Check out the initialization scripts for this tutorial on GitHub.
- Set Up MySQL on Compute Engine.
- Read about high availability for MySQL.
- Configure Cloud Logging for MySQL logs.
- Explore reference architectures, diagrams, and best practices about Google Cloud. Take a look at our Cloud Architecture Center.