Running the Java Bookshelf app on Compute Engine

This tutorial shows how to run the Java Bookshelf sample app on Compute Engine. Follow this tutorial to deploy an existing Java web app to Compute Engine. It's recommended to work through the Bookshelf app documentation as part of the tutorial for the App Engine standard environment.

Objectives

  • Deploy the Bookshelf sample app to a single Compute Engine instance.
  • Scale the app horizontally by using a managed instance group.
  • Serve traffic by using HTTP load balancing.
  • Respond to traffic changes by using autoscaling.

Costs

This tutorial uses the following billable components of Google Cloud Platform:

You can use the pricing calculator to generate a cost estimate based on your projected usage. New GCP users might be eligible for a free trial.

When you finish this tutorial, you can avoid continued billing by deleting the resources you created. For more information, see Cleaning up.

Before you begin

  1. Sign in to your Google Account.

    If you don't already have one, sign up for a new account.

  2. Select or create a Google Cloud Platform project.

    Go to the Manage resources page

  3. Make sure that billing is enabled for your Google Cloud Platform project.

    Learn how to enable billing

  4. Enable the Cloud Datastore API, Cloud Storage API, and Cloud Pub/Sub API APIs.

    Enable the APIs

  5. Install and initialize the Cloud SDK.
  6. Install Java 8 and Maven.

Initializing Cloud Datastore

The Bookshelf sample app uses Cloud Datastore to store the books. To initialize Cloud Datastore in your project for the first time, follow these steps:

  1. In the Google Cloud Platform Console, open Cloud Datastore.

    Open Cloud Datastore

  2. Select a region for your datastore, and then click Continue.

  3. When you reach the Create an Entity page, close the window. The Bookshelf app is ready to create entities on Cloud Datastore.

Creating a Cloud Storage bucket

The following instructions show how to create a Cloud Storage bucket. Buckets are the basic containers that hold your data in Cloud Storage.

  1. In your terminal window, enter the following command:

    gsutil mb gs://[YOUR_BUCKET_NAME]

    [YOUR_BUCKET_NAME] represents the name of your Cloud Storage bucket.

  2. To view uploaded images in the Bookshelf app, set the bucket's default access control list (ACL) to public-read:

    gsutil defacl set public-read gs://[YOUR_BUCKET_NAME]

Cloning the sample app

The sample app is available on GitHub at GoogleCloudPlatform/getting-started-java.

  1. Clone the repository.

    git clone https://github.com/GoogleCloudPlatform/getting-started-java.git
    
  2. Go to the sample directory.

    cd getting-started-java/bookshelf/6-gce
    

Configuring the app

  1. Open the makeBookshelf file for editing.
  2. Set the BUCKET variable with the name of the bucket you created previously.
  3. Optionally, you can change ZONE=us-central1-f to a different zone.
  4. Make the script executable:

    chmod +x makeBookshelf
    
  5. Open pom.xml for editing.

  6. Set the bookshelf.bucket property with the name of your bucket.

  7. Save and close both files.

Running the app on your local computer

  1. Start a local web server:

    mvn -Plocal clean jetty:run-exploded -DprojectID=[YOUR-PROJECT-ID]
    

    where [YOUR-PROJECT-ID] is your project ID.

  2. In your web browser, enter this address:

    http://localhost:8080

To stop the local web server, press Control+C.

Deploying to a single instance

Single-instance deployment

This section walks you through running a single instance of your app on Compute Engine.

Push your code to a repository

You can use Cloud Source Repositories to create a Git repository in your project and upload your app's code there. Your instances can then pull the latest version of your app's code from the repository during startup. Using a Git repository is convenient because updating your app doesn't require configuring new images or instances; just restart an existing instance or create one.

  • To create a Compute Engine instance and push your app to the instance, enter the following command:

    ./makeBookshelf gce
    

    The command does the following:

    • Builds the project and creates a Java WAR file.
    • Uploads the WAR file and some scripts to the Cloud Storage bucket that you created previously to store images.
    • Creates a Compute Engine instance and enables it to access Logging, Cloud Storage, and Cloud Datastore.
    • Passes metadata about your Cloud Storage bucket.
    • Specifies the startup script.
    mvn clean package
    
    gsutil cp -r target/${WAR} gce/base gs://${BUCKET}/gce/
    
    gcloud compute firewall-rules create allow-http-bookshelf \
      --allow tcp:80 \
      --source-ranges 0.0.0.0/0 \
      --target-tags ${TAGS} \
      --description "Allow port 80 access to instances tagged with ${TAGS}"
    
    gcloud compute instances create my-app-instance \
      --machine-type=${MACHINE_TYPE} \
      --scopes=${SCOPES} \
      --metadata-from-file startup-script=${STARTUP_SCRIPT} \
      --zone=${ZONE} \
      --tags=${TAGS} \
      --image-family=${IMAGE_FAMILY} \
      --image-project=${IMAGE_PROJECT} \
      --metadata BUCKET=${BUCKET}

    The makeBookshelf script defines several variables:

    ZONE=us-central1-f
    
    GROUP=frontend-group
    TEMPLATE=$GROUP-tmpl
    MACHINE_TYPE=g1-small
    IMAGE_FAMILY=debian-8
    IMAGE_PROJECT=debian-cloud
    STARTUP_SCRIPT=gce/startup-script.sh
    SCOPES="datastore,userinfo-email,logging-write,storage-full,cloud-platform"
    TAGS=http-server
    
    MIN_INSTANCES=1
    MAX_INSTANCES=10
    TARGET_UTILIZATION=0.6
    
    SERVICE=frontend-web-service
    WAR=bookshelf-1.0-SNAPSHOT.war
  • The instance takes about five minutes to run the startup script. You can check the progress of the instance by entering the following command:

    gcloud compute instances get-serial-port-output my-app-instance --zone us-central1-f
    
  • After the startup script completes, you can check the startup logs for your new instance at /var/logs/daemon.log.

  • The instance name is my-app-instance. You can obtain the external IP address of your instance by entering the following command:

    gcloud compute instances list
    
  • To see your app running, enter this URL in your browser:

    http://[YOUR_INSTANCE_IP]
    
  • To delete your instance, enter this command:

    ./makeBookshelf down
    

Use a startup script to initialize an instance

Now that Compute Engine instances can access your code, you need a way to instruct your instance to download and run your code. An instance can have a startup script that runs whenever the instance is started or restarted.

The startup script performs these tasks:

  • Installs Java 8 and makes it the default version.

  • Installs and configures Jetty.

  • Copies the Java WAR file from the Cloud Storage bucket to Jetty's webapps and renames it root.war. This makes it the root servlet, so it doesn't need to be named in the URL.

  • Installs the Logging agent and configures it to monitor the app's logs. This means that the logging configured in the previous steps of this tutorial are uploaded just as if you were using the App Engine flexible environment.

set -e
set -v

# Talk to the metadata server to get the project id
PROJECTID=$(curl -s "http://metadata.google.internal/computeMetadata/v1/project/project-id" -H "Metadata-Flavor: Google")
BUCKET=$(curl -s "http://metadata.google.internal/computeMetadata/v1/instance/attributes/BUCKET" -H "Metadata-Flavor: Google")

echo "Project ID: ${PROJECTID}  Bucket: ${BUCKET}"

# get our file(s)
gsutil cp "gs://${BUCKET}/gce/"** .

# Install dependencies from apt
apt-get update
apt-get install -t jessie-backports -yq openjdk-8-jdk

# Make Java8 the default
update-alternatives --set java /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java

# Jetty Setup
mkdir -p /opt/jetty/temp
mkdir -p /var/log/jetty

# Get Jetty
curl -L https://repo1.maven.org/maven2/org/eclipse/jetty/jetty-distribution/9.3.8.v20160314/jetty-distribution-9.3.8.v20160314.tar.gz -o jetty9.tgz
tar xf jetty9.tgz  --strip-components=1 -C /opt/jetty

# Add a Jetty User
useradd --user-group --shell /bin/false --home-dir /opt/jetty/temp jetty

cd /opt/jetty
# Add running as "jetty"
java -jar /opt/jetty/start.jar --add-to-startd=setuid
cd /

# very important - by renaming the war to root.war, it will run as the root servlet.
mv bookshelf-1.0-SNAPSHOT.war /opt/jetty/webapps/root.war

# Make sure "jetty" owns everything.
chown --recursive jetty /opt/jetty

# Configure the default paths for the Jetty service
cp /opt/jetty/bin/jetty.sh /etc/init.d/jetty
echo "JETTY_HOME=/opt/jetty" > /etc/default/jetty
{
  echo "JETTY_BASE=/opt/jetty"
  echo "TMPDIR=/opt/jetty/temp"
  echo "JAVA_OPTIONS=-Djetty.http.port=80"
  echo "JETTY_LOGS=/var/log/jetty"
} >> /etc/default/jetty

# -Dorg.eclipse.jetty.util.log.class=org.eclipse.jetty.util.log.JavaUtilLog

# Reload daemon to pick up new service
systemctl daemon-reload

# Install logging monitor. The monitor will automatically pickup logs sent to syslog.
curl -s "https://storage.googleapis.com/signals-agents/logging/google-fluentd-install.sh" | bash
service google-fluentd restart &

service jetty start
service jetty check

echo "Startup Complete"

Create and configure a Compute Engine instance

  1. Create a Compute Engine instance. This command creates an instance, allows it to access GCP services, and runs your startup script. The instance name is my-app-instance.

    Linux/macOS

    gcloud compute instances create my-app-instance \
        --image-family=debian-9 \
        --image-project=debian-cloud \
        --machine-type=g1-small \
        --scopes userinfo-email,cloud-platform \
        --metadata app-location=$BOOKSHELF_DEPLOY_LOCATION \
        --metadata-from-file startup-script=gce/startup-script.sh \
        --zone us-central1-f \
        --tags http-server
    

    Windows

    gcloud compute instances create my-app-instance ^
        --image-family=debian-9 ^
        --image-project=debian-cloud ^
        --machine-type=g1-small ^
        --scopes userinfo-email,cloud-platform ^
        --metadata-from-file startup-script=gce/startup-script.sh ^
        --zone us-central1-f ^
        --tags http-server
    

  2. Check the progress of the instance creation.

    gcloud compute instances get-serial-port-output my-app-instance --zone us-central1-f
    

    When the startup script completes, the output displays Finished running startup script.

  3. Create a firewall rule to allow traffic to your instance.

    Linux/macOS

    gcloud compute firewall-rules create default-allow-http-8080 \
        --allow tcp:8080 \
        --source-ranges 0.0.0.0/0 \
        --target-tags http-server \
        --description "Allow port 8080 access to http-server"
    

    Windows

    gcloud compute firewall-rules create default-allow-http-8080 ^
        --allow tcp:8080 ^
        --source-ranges 0.0.0.0/0 ^
        --target-tags http-server ^
        --description "Allow port 8080 access to http-server"
    

  4. Get the external IP address of your instance.

    gcloud compute instances list
    
  5. To see the app running, go to http://[YOUR_INSTANCE_IP]:8080,

    where [YOUR_INSTANCE_IP] is the external IP address of your instance.

Manage and monitor an instance

You can use the GCP Console to monitor and manage your instance.

  • To view the running instance and connect to it by using ssh, in GCP Console, go to the VM instances page.

    Go to the VM instances page

  • To view all of the logs generated by your Compute Engine resources, in GCP Console, go to the Logs page.

    Go to the Logs page

    Logging is automatically configured to gather logs from various common services, including syslog.

Horizontal scaling with multiple instances

Multiple-instance deployment with managed instances

Compute Engine can scale horizontally. By using a managed instance group and the Compute Engine Autoscaler, Compute Engine can automatically create instances of your app when needed and shut down instances when demand is low. You can set up an HTTP load balancer to distribute traffic to the instances in a managed instance group.

Deployment script

The sample app includes a script that automates the following deployment steps. The script named deploy.sh deploys the resources for a complete, autoscaled, load-balanced app as described in Horizontal scaling with multiple instances.

You can run each of the following steps yourself, or run gce/deploy.sh from the gce directory.

Create a managed instance group

A managed instance group is a group of homogeneous instances based on the same instance template. An instance template defines the configuration of your instance, including source image, disk size, scopes, and metadata (including startup scripts).

  1. Create a template.

    ./makeBookshelf gce-many
    
  2. Create an instance group.

    gcloud compute instance-groups managed \
      create ${GROUP} \
      --base-instance-name ${GROUP} \
      --size ${MIN_INSTANCES} \
      --template ${TEMPLATE} \
      --zone ${ZONE}

    The --size parameter specifies the number of instances in the group. When all of the instances finish running their startup scripts, you can access the instances individually by using their external IP addresses and port 8080. To find the external IP addresses of the instances, enter gcloud compute instances list. The managed instances have names that start with the same prefix, my-app, which you specified in the --base-instance-name parameter.

Create a load balancer

An individual instance is fine for testing or debugging, but for serving web traffic it's better to use a load balancer to automatically direct traffic to available instances.

  1. Create a health check.

    The load balancer uses a health check to determine which instances are capable of serving traffic.

    gcloud compute http-health-checks create ah-health-check \
      --request-path /_ah/health \
      --port 80
  2. Create a named port.

    The HTTP load balancer looks for the http service to know which port to direct traffic to. In your existing instance group, give port 8080 the name http.

  3. Create a backend service.

    The backend service is the target for load-balanced traffic. It defines which instance group the traffic is directed to and which health check to use.

    gcloud compute backend-services create $SERVICE \
      --http-health-checks ah-health-check
  4. Add the backend service.

    gcloud compute backend-services add-backend $SERVICE \
      --instance-group $GROUP \
      --zone $ZONE
  5. Create a URL map and proxy.

    The URL map defines which URLs are directed to which backend services. In this sample, all traffic is served by one backend service. If you want to load balance requests between multiple regions or groups, you can create multiple backend services. A proxy receives traffic and forwards it to backend services by using URL maps.

    1. Create the URL map.

      gcloud compute url-maps create $SERVICE-map \
        --default-service $SERVICE
    2. Create the proxy.

      gcloud compute target-http-proxies create $SERVICE-proxy \
        --url-map $SERVICE-map
  6. Create a global forwarding rule. The global forwarding rule ties a public IP address and port to a proxy.

    gcloud compute forwarding-rules create $SERVICE-http-rule \
      --global \
      --target-http-proxy $SERVICE-proxy \
      --port-range 80

Configure the autoscaler

The load balancer ensures that traffic is distributed across all of your healthy instances. But what happens if there is too much traffic for your instances to handle? You could manually add more instances. But a better solution is to configure a Compute Engine autoscaler to automatically create and delete instances in response to traffic demands.

  1. Create an autoscaler.

    gcloud compute instance-groups managed set-autoscaling \
      $GROUP \
      --max-num-replicas $MAX_INSTANCES \
      --target-load-balancing-utilization $TARGET_UTILIZATION \
      --zone $ZONE

    The preceding command creates an autoscaler on the managed instance group that automatically scales up to 10 instances. Instances are added when the load balancer is above 50% utilization and are removed when utilization falls below 50%.

  2. Create a firewall rule.

    Linux/macOS

    gcloud compute firewall-rules create default-allow-http-8080 \
        --allow tcp:8080 \
        --source-ranges 0.0.0.0/0 \
        --target-tags http-server \
        --description "Allow port 8080 access to http-server"
    

    Windows

    gcloud compute firewall-rules create default-allow-http-8080 ^
        --allow tcp:8080 ^
        --source-ranges 0.0.0.0/0 ^
        --target-tags http-server ^
        --description "Allow port 8080 access to http-server"
    

  3. Check progress until at least one of your instances reports HEALTHY.

    gcloud compute backend-services get-health frontend-web-service --global
    

View your app

  1. Get the forwarding IP address for the load balancer.

    gcloud compute forwarding-rules list --global
    

    Your forwarding-rules IP address is in the IP_ADDRESS column.

  2. In a browser, enter the IP address from the list.

    Your load-balanced and autoscaled app is now running on Compute Engine.

Manage and monitor your deployment

You can use the GCP Console to monitor load balancing, autoscaling, and your managed instance group.

  • You can monitor and manage your instance group and autoscaling configuration on the Instance groups page.

    Go to the Instance groups page

  • You can monitor and manage your load balancing configuration, including URL maps and backend services, on the Load balancing page.

    Go to the Load balancing page

Cleaning up

To avoid incurring charges to your Google Cloud Platform account for the resources used in this tutorial:

Run the teardown script

If you ran the deploy.sh script, run the teardown.sh script to remove all resources created by the deploy.sh script. This returns your project to the state before running the deploy.sh script and helps to avoid further billing. To remove the single instance and the storage bucket created at the beginning of the tutorial, follow the instructions in the next section.

Delete resources manually

If you followed the steps in this tutorial manually, you can manually delete the cloud resources that you created.

Delete your load balancer

  1. In the GCP Console, go to the Load Balancing page.

    Go to the Load Balancing page

  2. Select the checkbox next to the load balancer that you want to delete, and then click Deletedelete.

  3. In the Delete load balancer dialog, select the associated backend service and health check resources, and then click Deletedelete. The load balancer and its associated resources are deleted.

Delete your Compute Engine managed instance group

To delete a Compute Engine instance group:

  1. In the GCP Console, go to the Instance groups page.

    Go to the Instance groups page

  2. Click the checkbox for the instance group you want to delete.
  3. Click Delete to delete the instance group.

Delete your single Compute Engine instance

To delete a Compute Engine instance:

  1. In the GCP Console, go to the VM Instances page.

    Go to the VM Instances page

  2. Click the checkbox for the instance you want to delete.
  3. Click Delete to delete the instance.

Delete your Cloud Storage bucket

To delete a Cloud Storage bucket:

  1. In the GCP Console, go to the Cloud Storage Browser page.

    Go to the Cloud Storage Browser page

  2. Click the checkbox for the bucket you want to delete.
  3. Click Delete to delete the bucket.

What's next

Was this page helpful? Let us know how we did:

Send feedback about...