This tutorial shows how to run the Python Bookshelf app on Google Compute Engine. Follow this tutorial to deploy an existing Python web app to Compute Engine. You don't have to be familiar with the Bookshelf app to follow this tutorial, but if you would like to learn about the Bookshelf app, see the tutorial for the App Engine flexible environment.
- Deploy the Bookshelf sample app to a single Compute Engine instance.
- Scale the app horizontally by using a managed instance group.
- Serve traffic by using HTTP load balancing.
- Respond to traffic changes by using autoscaling.
This tutorial uses billable components of Cloud Platform, including:
- Google Compute Engine
- Google Cloud Storage
- Google Cloud Datastore
- Google Cloud Logging
- Google Cloud Pub/Sub
Before you begin
Sign in to your Google Account.
If you don't already have one, sign up for a new account.
Select or create a GCP project.
Make sure that billing is enabled for your project.
- Enable the Cloud Datastore, Cloud Storage, and Cloud Pub/Sub APIs.
- Install and initialize the Cloud SDK.
- Install Python, pip, and virtualenv on your system. For instructions, refer to Setting Up a Python Development Environment for Google Cloud Platform.
Creating a Cloud Storage bucket
The following instructions show how to create a Cloud Storage bucket. Buckets are the basic containers that hold your data in Cloud Storage.
To create a bucket:
Invoke the following command in a terminal window:
gsutil mb gs://[YOUR-BUCKET-NAME]
Set the bucket's default ACL to
public-read, which enables users to see their uploaded images:
gsutil defacl set public-read gs://[YOUR-BUCKET-NAME]
Cloning the sample app
The sample application is available on GitHub at GoogleCloudPlatform/getting-started-python.
Clone the repository:
git clone https://github.com/GoogleCloudPlatform/getting-started-python.git
Go to the sample directory:
Configuring the app
Set the value of
PROJECT_IDto your project ID.
Set the value CLOUD_STORAGE_BUCKET to your Cloud Storage bucket name.
Save and close
Running the app on your local computer
Create an isolated Python environment, and install dependencies:
virtualenv env source env/bin/activate pip install -r requirements.txt
virtualenv env env\scripts\activate pip install -r requirements.txt
Run the application:
In your web browser, enter this address:
To stop the local web server, press Control+C.
Deploying to a single instance
This section walks you through running a single instance of your application on Compute Engine.
Pushing your code to a repository
You can use Cloud Source Repositories to easily create a Git repository in your project and upload your application code there. Your instances can then pull the latest version of your application code from the repository during startup. This is convenient because updating your application does not require configuring new images or instances; all you need to do is restart an existing instance or create a new one.
First, create an repository here on your Cloud Console.
Then push your application code to your project's repository:
git commit -am "Updating configuration" git config credential.helper gcloud.sh gcloud beta source repos create [YOUR_REPO] git remote add cloud https://source.developers.google.com/p/[YOUR_PROJECT_ID]/r/[YOUR_REPO] git push cloud
[YOUR_PROJECT_ID] is your project ID and
[YOUR_REPO] is the name you
given to the repository you just created.
Using a startup script to initialize an instance
Now that your code is accessible by Compute Engine instances, you need a way to instruct your instance to download and run your code. An instance can have a startup script that is executed whenever the instance is started or restarted.
Here is the startup script that is included in the Bookshelf sample app:
set -v # Talk to the metadata server to get the project id PROJECTID=$(curl -s "http://metadata.google.internal/computeMetadata/v1/project/project-id" -H "Metadata-Flavor: Google") # Install logging monitor. The monitor will automatically pickup logs sent to # syslog. curl -s "https://storage.googleapis.com/signals-agents/logging/google-fluentd-install.sh" | bash service google-fluentd restart & # Install dependencies from apt apt-get update apt-get install -yq \ git build-essential supervisor python python-dev python-pip libffi-dev \ libssl-dev # Create a pythonapp user. The application will run as this user. useradd -m -d /home/pythonapp pythonapp # pip from apt is out of date, so make it update itself and install virtualenv. pip install --upgrade pip virtualenv # Get the source code from the Google Cloud Repository # git requires $HOME and it's not set during the startup script. export HOME=/root git config --global credential.helper gcloud.sh git clone https://source.developers.google.com/p/$PROJECTID/r/[YOUR_REPO_NAME] /opt/app # Install app dependencies virtualenv -p python3 /opt/app/7-gce/env source /opt/app/7-gce/env/bin/activate /opt/app/7-gce/env/bin/pip install -r /opt/app/7-gce/requirements.txt # Make sure the pythonapp user owns the application code chown -R pythonapp:pythonapp /opt/app # Configure supervisor to start gunicorn inside of our virtualenv and run the # application. cat >/etc/supervisor/conf.d/python-app.conf << EOF [program:pythonapp] directory=/opt/app/7-gce command=/opt/app/7-gce/env/bin/honcho start -f ./procfile worker bookshelf autostart=true autorestart=true user=pythonapp # Environment variables ensure that the application runs inside of the # configured virtualenv. environment=VIRTUAL_ENV="/opt/app/7-gce/env",PATH="/opt/app/7-gce/env/bin",\ HOME="/home/pythonapp",USER="pythonapp" stdout_logfile=syslog stderr_logfile=syslog EOF supervisorctl reread supervisorctl update # Application should now be running under supervisor
To use this script, replace
[YOUR_REPO_NAME] with the name of your repository.
The startup script performs these tasks:
Install the Google Cloud Logging agent. The agent automatically collects logs from syslog.
Install Python and Supervisor. Supervisor is used to run the application as a daemon.
Clone the application source code from the Cloud Repository and installs dependencies.
Configure Supervisor to run the application. Supervisor makes sure the application is restarted if it exits unexpectedly or is killed by an administrator or other process. It also sends the application's
stderrto syslog to be collected by the Cloud Logging agent.
Creating and configuring a Compute Engine instance
Create a Compute Engine instance:
gcloud compute instances create my-app-instance \ --image-family=debian-8 \ --image-project=debian-cloud \ --machine-type=g1-small \ --scopes userinfo-email,cloud-platform \ --metadata app-location=$BOOKSHELF_DEPLOY_LOCATION \ --metadata-from-file startup-script=gce/startup-script.sh \ --zone us-central1-f \ --tags http-server
gcloud compute instances create my-app-instance ^ --image-family=debian-8 ^ --image-project=debian-cloud ^ --machine-type=g1-small ^ --scopes userinfo-email,cloud-platform ^ --metadata-from-file startup-script=gce/startup-script.sh ^ --zone us-central1-f ^ --tags http-server
This command creates a new instance, allows it to access Cloud Platform services, and runs your startup script. The instance name is
Check the progress of the instance creation:
gcloud compute instances get-serial-port-output my-app-instance --zone us-central1-f
If the startup script has completed, you will see
Finished running startup scriptnear the end of the command output.
Create a firewall rule to allow traffic to your instance:
gcloud compute firewall-rules create default-allow-http-8080 \ --allow tcp:8080 \ --source-ranges 0.0.0.0/0 \ --target-tags http-server \ --description "Allow port 8080 access to http-server"
gcloud compute firewall-rules create default-allow-http-8080 ^ --allow tcp:8080 ^ --source-ranges 0.0.0.0/0 ^ --target-tags http-server ^ --description "Allow port 8080 access to http-server"
Get the external IP address of your instance:
gcloud compute instances list
To see the application running, go to
[YOUR_INSTANCE_IP]is the external IP address of your instance.
Managing and monitoring an instance
You can use the Google Cloud Platform Console to monitor and manage your instance. In the Compute > Compute Engine section, you can view the running instance and connect to it using SSH. In the Monitoring > Logs section, you can view all of the logs generated by your Compute Engine resources. Google Cloud Logging is automatically configured to gather logs from various common services, including syslog.
Horizontal scaling with multiple instances
Compute Engine can easily scale horizontally. By using a managed instance group and the Compute Engine Autoscaler, Compute Engine can automatically create new instances of your application when needed and shut down instances when demand is low. You can set up an HTTP load balancer to distribute traffic to the instances in a managed instance group.
Creating a managed instance group
A managed instance group is a group of homogeneous instances based on the same instance template. An instance template defines the configuration of your instance, including source image, disk size, scopes, and metadata, including startup scripts.
First, create a template:
gcloud compute instance-templates create $TEMPLATE \ --image-family $IMAGE_FAMILY \ --image-project $IMAGE_PROJECT \ --machine-type $MACHINE_TYPE \ --scopes $SCOPES \ --metadata-from-file startup-script=$STARTUP_SCRIPT \ --tags $TAGS
Next, create an instance group:
gcloud compute instance-groups managed \ create $GROUP \ --base-instance-name $GROUP \ --size $MIN_INSTANCES \ --template $TEMPLATE \ --zone $ZONE
--sizeparameter species the number of instances in the group. After all of the instances have finished running their startup scripts, the instances can be accessed individually by using their external IP addresses and port 8080. To find the external IP addresses of the instances, enter
gcloud compute instances list. The managed instances have names that start with the same prefix,
my-app, which you specified in the
Creating a load balancer
An individual instance is fine for testing or debugging, but for serving web traffic it's better to use a load balancer to automatically direct traffic to available instances. To create a load balancer, follow these steps.
Create a health check. The load balancer uses a health check to determine which instances are capable of serving traffic:
gcloud compute http-health-checks create ah-health-check \ --request-path /_ah/health \ --port 8080
Create a named port. The HTTP load balancer looks for the
httpservice to know which port to direct traffic to. In your existing instance group, give port 8080 the name
gcloud compute instance-groups managed set-named-ports \ $GROUP \ --named-ports http:8080 \ --zone $ZONE
Create a backend service. The backend service is the target for load-balanced traffic. It defines which instance group the traffic should be directed to and which health check to use.
gcloud compute backend-services create $SERVICE \ --http-health-checks ah-health-check \ --global
Add the backend service:
gcloud compute backend-services add-backend $SERVICE \ --instance-group $GROUP \ --instance-group-zone $ZONE \ --global
Create a URL map and proxy:
The URL map defines which URLs should be directed to which backend services. In this sample, all traffic is served by one backend service. If you want to load balance requests between multiple regions or groups, you can create multiple backend services. A proxy receives traffic and forwards it to backend services using URL maps.
Create the URL map:
gcloud compute url-maps create $SERVICE-map \ --default-service $SERVICE
Create the proxy:
gcloud compute target-http-proxies create $SERVICE-proxy \ --url-map $SERVICE-map
Create a global forwarding rule. The global forwarding rule ties a public IP address and port to a proxy:
gcloud compute forwarding-rules create $SERVICE-http-rule \ --global \ --target-http-proxy $SERVICE-proxy \ --ports=80
Configuring the autoscaler
The load balancer ensures that traffic is distributed across all of your healthy instances. But what happens if there is too much traffic for your instances to handle? You could manually add more instances. But a better solution is to configure a Compute Engine Autoscaler to automatically create and delete instances in response to traffic demands.
Create an autoscaler:
gcloud compute instance-groups managed set-autoscaling \ $GROUP \ --max-num-replicas $MAX_INSTANCES \ --target-load-balancing-utilization $TARGET_UTILIZATION \ --zone $ZONE
The preceding command creates an autoscaler on the managed instance group that automatically scales up to 10 instances. New instances are added when the load balancer is above 50% utilization and are removed when utilization falls below 50%.
Create a firewall rule if you haven't already created one:
# Check if the firewall rule has been created in previous steps of the documentation if gcloud compute firewall-rules list --filter="name~'default-allow-http-8080'" \ --format="table(name)" | grep -q 'NAME'; then echo "Firewall rule default-allow-http-8080 already exists." else gcloud compute firewall-rules create default-allow-http-8080 \ --allow tcp:8080 \ --source-ranges 0.0.0.0/0 \ --target-tags http-server \ --description "Allow port 8080 access to http-server" fi
gcloud compute backend-services get-health my-app-service
Continue checking until at least one of your instances reports
Viewing your application
Get the forwarding IP address for the load balancer:
gcloud compute forwarding-rules list --global
Your forwarding-rules IP address is in the
In a browser, enter the IP address from the list. Your load-balanced and autoscaled app is now running on Google Compute Engine!
Managing and monitoring your deployment
Managing multiple instances is just as easy as managing a single instance. You can use the GCP Console to monitor load balancing, autoscaling, and your managed instance group.
You can manage your instance group and autoscaling configuration by using the Compute > Compute Engine > Instance groups section.
You can manage your load balancing configuration, including URL maps and backend services, by using the Compute > Compute Engine > HTTP load balancing section.
The sample application includes a script that helps demonstrate deployment to
Compute Engine. The script named
deploy.sh performs a complete, autoscaled,
load-balanced deployment of the application as described in
Horizontal scaling with multiple instances.
To avoid incurring charges to your Google Cloud Platform account for the resources used in this tutorial:
Running the teardown script
If you ran the
deploy.sh script, run the
teardown.sh script to remove all
resources created by the
deploy.sh script. This returns your project to the
state before running
deploy.sh script and helps to avoid further billing. To
remove the single instance and the storage bucket created at the beginning of
the tutorial, follow the instructions in the next section.
Deleting resources manually
If you followed the steps in this tutorial manually, you can manually delete the cloud resources you created.
Deleting your load balancer
To delete a load balancer:
In the GCP Console, go to the Load Balancing page.
Click the checkbox next to the load balancer you want to delete.
Click the Delete button at the top of the page to delete the load balancer.
In the Delete load balancer dialog, select the associated backend service and health check resources.
Click the Delete Load Balancer button to delete the load balancer and its associated resources.
Deleting your Compute Engine managed instance group
To delete a Compute Engine instance group:
- In the GCP Console, go to the Instances Groups page.
- Click the checkbox next to the instance group you want to delete.
- Click the Delete button at the top of the page to delete the instance group.
Deleting your single Compute Engine instance
To delete a Compute Engine instance:
- In the GCP Console, go to the VM Instances page.
- Click the checkbox next to the instance you want to delete.
- Click the Delete button at the top of the page to delete the instance.
Deleting your Cloud Storage bucket
To delete a Cloud Storage bucket:
- In the GCP Console, go to the Cloud Storage browser.
- Click the checkbox next to the bucket you want to delete.
- Click the Delete button at the top of the page to delete the bucket.
- Learn how to run the Python Bookshelf sample on Kubernetes Engine.
Try out other Google Cloud Platform features for yourself. Have a look at our tutorials.
Explore other Cloud Platform services.