Deploying the Cloud Spanner Emulator locally

Welcome to the second part of our series on the Cloud Spanner Emulator. In the first part, we got an overview of Cloud Spanner and the emulator, as well as the various ways that it can be provisioned. 

Today, we will learn about running the Cloud Spanner emulator locally and containerizing + deploying the backend of our sample application (OmegaTrade) on the local emulator. 

Starting the emulator locally 

The emulator has two processes, emulator_main which contains a gRPC server and gateway_main which is the REST gateway that also starts emulator_main as a subprocess. If your application only uses client libraries or the RPC API, you can just start the emulator_main process via Linux prebuilt binaries. If your application uses the REST API, then you should start gateway_main. The default ports are 9010 and 9020 for the gRPC server and REST server respectively, but they can be changed while starting the emulator via the docker / Linux pre-built libraries.

There are various options to start the emulator locally. Here we will cover the gcloud instructions, but instructions for other methods can be found within the Cloud Spanner emulator GitHub repository. 

We begin by updating our gcloud components and starting the emulator:

NOTE - If you are getting an error like:

You can try the following commands (or their equivalents for your operating system): 

NOTE - This command starts the emulator in the background. To stop the emulator_main and gateway_main processes on a Linux/UNIX machine you may execute the following commands

Let’s create a configuration for the emulator. This is a one-time setup and can be reused subsequently. Run the following commands:

In your development environment, you might want to switch between using a local emulator or connecting to a production Cloud Spanner instance. You can manage this by having multiple gcloud configurations and switching between configurations by using the following command

Let’s create an instance, database and some tables on the local emulator. 

Create an instance in the emulator

Create a database

Create tables

Verify that the tables were successfully created by querying INFORMATION_SCHEMA in the emulator instance

Now that the emulator is up and running, let’s clone the sample app repo and deploy the backend service of OmegaTrade with the emulator backend.

Create the .env file and ensure the project id, instance name and database name match the ones we created above.

Build and run the application

The app should be up and running on http://localhost:3000

The OmegaTrade app is using the Node.js Cloud Spanner client and interacting with the emulator via the gRPC endpoint. 

Let’s make a curl request to insert some data into the emulator via the OmegaTrade app

Verify if the data was inserted with a query

The output should look something like this:

Using the emulator with containerized applications

Bundling the emulator as one of the application components

While developing and testing containerized applications, you can either run an emulator as a standalone container locally or point to a remote host where the emulator is running using its public IP or hostname in the gcloud configuration. 

NOTE - Ensure that your firewall allows ingress TCP traffic on ports 9010 and 9020.

Example:

NOTE - If you are running the emulator on a remote host, this will send data from the client (local) to the emulator (remote) over the internet as plain text. That is not a problem for test data, but if you are using this for testing/development/debugging, you should be careful to use only synthetic data and not privacy-sensitive data.

Another option is to build and execute the emulator as one of the components of the application image. We will do that now, by building a new Docker image with both the application code and the emulator, creating tables and then running the service.

Take a look at dockerfile.local.emulator here. You will notice that we are using the prebuilt emulator docker image from GCR as the static source to copy emulator libraries to our image. Alternatively, you can also run wget to download these binaries in your Docker container. Since the OmegaTrade backend is a Node.js app, we will start with google/cloud-sdk:slim as the base image and install Cloud SDK using a RUN command. Finally, the init script start_spanner.bash here does quite a few things: it starts the emulator, creates an instance, database and tables on the local emulator, starts the application, and acts as the entry point. 

Before building and deploying the Docker image, we need to create a .env file with the environment variables that we need for running the application.

Note - The project ID in the .env file must be the same as in the start_spanner.bash file.

Let’s build and deploy

After running the ‘docker run’ command, wait for 15-20 seconds while everything is being set up in the container.

You can verify the functionality of the app by inserting some data via the application into the emulator

Again, you can verify if the data was inserted with a query

Running a standalone emulator and deploying the app

We can run the app and the emulator in separate containers and test the application locally. In order to do so, we need to create a Docker network and start the application and emulator containers attached to this network. Take a look at dockerfile.standalone.emulator here. The start_spanner_standalone.bash script here acts as the entry point, which connects to the emulator and creates an instance, database and schema. Now, let’s go ahead and deploy. 

Create a Docker network

Start the emulator in a container named “emulator” (because the app container will connect to the emulator by this name) and specify the network

If you’ve already followed the steps in the previous section, you may skip cloning the repository and creating the .env file below.

Before building and deploying the Docker image, we need to create a .env file with the environment variables that we need for running the application.

Note - The project ID in the .env file must be the same as in the start_spanner.bash file.

Start the application container, specifying the network we created

After running the ‘docker run’ command, you may need to wait for 15-20 seconds as everything is being set up in the container.

If you’re curious, you can view the logs via

You can now test the application with a curl command

In the next and final part of the series, we will learn about running the emulator on a remote GCE instance. Stay tuned!