Edit on GitHub
Report issue
Page history

Zero to LAMP deployment on GCP with Chef

Author(s): @slevenick ,   Published: 2018-12-17


Objectives

  • Create a Chef Workstation on Google Cloud Platform.
  • Use the Workstation to create a Google Compute Engine VM, Cloud SQL instance, firewall rules, and a database via Chef.
  • Deploy Apache and PHP on the VM to serve a simple page backed by the database instance.

Costs

This tutorial uses billable components of Google Cloud Platform, including:

Use the Pricing Calculator to generate a cost estimate based on your projected usage.

Before you begin

  1. Select or create a Google Cloud Platform Console project. Go to the projects page.
  2. Enable billing for your project. Enable billing.
  3. Install the Google Cloud SDK.
  4. Authenticate gcloud with Google Cloud Platform.

    gcloud init
    

You will be executing commands on both a local machine (assuming Mac or Linux-based) and a Google Compute Engine VM. Command blocks will be prefixed with the bash comment (#) to indicate the correct machine.

Create a Chef Workstation on a Compute Engine VM

  1. The instructions assume you are using either a Mac or Linux-based local machine. Specify default settings for the Cloud SDK. Future gcloud commands will assume you are using this project, region, and zone.

    # local-machine
    PROJECT_NAME=your-project-name-here
    gcloud config set project $PROJECT_NAME
    gcloud config set compute/region us-west1
    gcloud config set compute/zone us-west1-a
    gcloud services enable compute.googleapis.com
    gcloud services enable sqladmin.googleapis.com
    gcloud services enable iam.googleapis.com
    

Create the service accounts that Chef will use

From the original machine:

  1. Create Compute Engine service account

    1. Create a new service account

      # local-machine
      gcloud iam service-accounts create chef-workstation
      
    2. Add Compute Engine and Cloud SQL permissions to the account

      # local-machine
      gcloud projects add-iam-policy-binding $PROJECT_NAME \            
        --member='serviceAccount:chef-workstation@$PROJECT_NAME.iam.gserviceaccount.com' \
        --role='roles/compute.admin'
      gcloud projects add-iam-policy-binding $PROJECT_NAME \
        --member='serviceAccount:chef-workstation@$PROJECT_NAME.iam.gserviceaccount.com' \
        --role='roles/iam.serviceAccountAdmin'
      gcloud projects add-iam-policy-binding $PROJECT_NAME \
        --member='serviceAccount:chef-workstation@$PROJECT_NAME.iam.gserviceaccount.com' \
        --role='roles/iam.serviceAccountUser'
      gcloud projects add-iam-policy-binding $PROJECT_NAME \
        --member='serviceAccount:chef-workstation@$PROJECT_NAME.iam.gserviceaccount.com' \
        --role='roles/cloudsql.admin'
      gcloud projects add-iam-policy-binding $PROJECT_NAME \
        --member='serviceAccount:chef-workstation@$PROJECT_NAME.iam.gserviceaccount.com' \
        --role='roles/iam.serviceAccountKeyAdmin'
      
  2. Use gcloud to create a new Compute Engine instance with the service account you just created and ssh to it.

    # local-machine
    gcloud compute instances create chef-workstation \
      --image-family debian-9 --image-project debian-cloud \
      --service-account=chef-workstation@$PROJECT_NAME.iam.gserviceaccount.com \
      --scopes=https://www.googleapis.com/auth/cloud-platform
    gcloud compute ssh chef-workstation
    
  3. From the chef-workstation machine, install the Chef Workstation and git:

    # gce: chef-workstation
    PROJECT_NAME=$(gcloud compute project-info describe | grep ^name | awk '{print $2}')
    echo "PROJECT_NAME=$PROJECT_NAME" >> ~/.bashrc
    wget https://packages.chef.io/files/stable/chef-workstation/0.1.162/ubuntu/18.04/chef-workstation_0.1.162-1_amd64.deb
    sudo dpkg -i chef-workstation_0.1.162-1_amd64.deb
    sudo apt update
    sudo apt install git -y
    exit
    

Create an infrastructure cookbook

  1. Create a cookbooks folder.

    # gce: chef-workstation
    mkdir -p chef-repo/cookbooks
    cd chef-repo/cookbooks
    
  2. Chef requires a git repository with at least one commit, so set up git and make an initial commit in the cookbooks directory.

    # gce: chef-workstation
    git config --global user.email "youremail@provider.com"
    git config --global user.name "Your Name"
    git init
    git commit -am "Initial Commit" --allow-empty
    
  3. Generate the Chef cookbook to specify your infrastructure.

    # gce: chef-workstation
    chef generate cookbook infrastructure
    
  4. Add the dependency on the google-cloud cookbook to the metadata.rb file of the cookbook

    # gce: chef-workstation
    echo "depends 'google-cloud', '~> 0.4.0'" >> infrastructure/metadata.rb
    
  5. Install GCP cookbooks from the supermarket using the knife tool

    # gce: chef-workstation
    knife cookbook site install google-cloud -o .
    
  6. Create a copy of your chef-workstation service account key on the chef workstation machine. This will be used by the Chef cookbooks you create.

    # gce: chef-workstation
    gcloud iam service-accounts keys create ~/key.json --iam-account=chef-workstation@$PROJECT_NAME.iam.gserviceaccount.com
    

Create Compute Engine cookbook

Use the Chef GCP cookbooks to define your Compute Engine infrastructure. More information on these cookbooks can be found at the Chef Supermarket.

  1. Create a new recipe within the infrastructure cookbook.

    # gce: chef-workstation
    chef generate recipe infrastructure compute
    
  2. Create the credentials that will be used to communicate with the Compute Engine APIs within the new compute recipe within the infrastructure cookbook.

    gauth_credential 'compute-credentials' do
      action :serviceaccount
      path '/home/$USER/key.json'
      scopes [
        'https://www.googleapis.com/auth/cloud-platform'
      ]
    end
    
  3. Create a network to build our infrastructure in. This network will contain your Compute Engine instances and allow you to create firewall rules that apply to everything within the network.

    gcompute_network 'web-server-network' do
      action :create
      auto_create_subnetworks true
      project '$PROJECT_NAME'
      credential 'compute-credentials'
    end
    
  4. Add a firewall rule that allows ssh and http traffic for your web server.

    gcompute_firewall 'fw-allow-ssh-http' do
      action :create
      allowed [
        {
          ip_protocol: 'tcp',
          ports: ['22', '80']
        }
      ]
      network 'web-server-network'
      project '$PROJECT_NAME'
      credential 'compute-credentials'
    end
    
  5. Create the Compute Engine instance and static IP address.

    gcompute_address 'web-server-ip' do
      action :create
      region 'us-west1'
      project '$PROJECT_NAME'
      credential 'compute-credentials'
    end
    
    gcompute_disk 'web-server-os-1' do
      action :create
      source_image 'projects/debian-cloud/global/images/family/debian-9'
      zone 'us-west1-a'
      project '$PROJECT_NAME'
      credential 'compute-credentials'
    end
    
    gcompute_instance 'web-server' do
      action :create
      machine_type 'n1-standard-1'
      disks [
        {
          boot: true,
          auto_delete: true,
          source: 'web-server-os-1'
        }
      ]
      network_interfaces [
      {
        network: 'web-server-network',
          access_configs: [
            {
              name: 'External NAT',
              nat_ip: 'web-server-ip',
              type: 'ONE_TO_ONE_NAT'
            }
          ]
        }
      ]
      zone 'us-west1-a'
      project '$PROJECT_NAME'
      credential 'compute-credentials'
    end    
    
  6. Include the Compute Engine recipe to your infrastructure cookbook's default recipe.

    # gce: chef-workstation
    echo "include_recipe 'infrastructure::compute'" >> infrastructure/recipes/default.rb
    

Create Cloud SQL cookbook

Use the google-gsql GCP cookbook to define a Cloud SQL instance.

  1. Create a new recipe within the infrastructure cookbook.

    # gce: chef-workstation
    chef generate recipe infrastructure sql
    
  2. Create credentials that will be used for building the Cloud SQL infrastructure.

    gauth_credential 'sql-credentials' do
      action :serviceaccount
      path '/home/$USER/key.json'
      scopes [
        'https://www.googleapis.com/auth/sqlservice.admin'
      ]
    end
    
  3. Define the Cloud SQL instance. To allow access from your webserver you must add the IP address of the web server to the authorized_networks field of the instance. The gcompute_address_ip function allows you to dynamically find the address of a named static IP address.

    ::Chef::Resource.send(:include, Google::Functions)
    gsql_instance 'sql-instance' do
      action :create
      database_version 'MYSQL_5_7'
      settings({
        tier: 'db-n1-standard-1',
        ip_configuration: {
          authorized_networks: [
            name: 'webapp',
            value: gcompute_address_ip('web-server-ip', 'us-west1', '$PROJECT_NAME', gauth_credential_serviceaccount_for_function('/home/$USER/key.json', ['https://www.googleapis.com/auth/compute']))
          ]
        }
      })
      region 'us-west1'
      project '$PROJECT_NAME'
      credential 'sql-credentials'
    end
    
  4. Define the database and MySQL user to create.

    gsql_database 'my_db' do
      action :create
      charset 'utf8'
      instance "sql-instance"
      project '$PROJECT_NAME'
      credential 'sql-credentials'
    end
    
    gsql_user 'db_user' do
      action :create
      password 'S3cUr!tY'
      host '%'
      instance 'sql-instance'
      project '$PROJECT_NAME'
      credential 'sql-credentials'
    end
    
  5. Include the Cloud SQL recipe into your infrastructure cookbook's default recipe.

    # gce: chef-workstation
    echo "include_recipe 'infrastructure::sql'" >> infrastructure/recipes/default.rb
    

It is important to be sure that the sql cookbook is included after the compute cookbook because the sql cookbook depends on the web server IP address existing.

Run the infrastructure cookbook

Now that you have created your infrastructure cookbook, run it on your Chef Workstation to configure your cloud environment.

  1. Run chef-client in local mode with the infrastructure cookbook as the run-list.

    # gce: chef-workstation
    chef-client -z -o infrastructure
    

This may take a couple of minutes as it creates the virtual machine and Cloud SQL instance.

Create a web server cookbook

For the purposes of this tutorial you can download a LAMP cookbook that can be run on Debian 9 here. This cookbook is adapted from https://learn.chef.io/modules/create-a-web-app-cookbook to work within the context of chef-run and connect to an external MySQL server. Download this cookbook to your Chef Workstation and unzip it within the cookbooks directory that you created.

    # gce: chef-workstation
    $USER@chef-workstation:~/chef-repo/cookbooks$ ls
    google-cloud  google-gauth  google-gcompute  google-gcontainer  google-gdns
    google-glogging  google-gsql  google-gstorage  infrastructure  lamp

Configure the web server cookbook for your environment

  1. Find your Cloud SQL IP address using gcloud sql instances list from your original machine.

  2. Modify the lamp/attributes/default.rb file to match your environment. Be sure to update default['lamp']['database']['dbhost'] = '$CLOUD_SQL_IP' to reference the IP address of your Cloud SQL instance.

Run the cookbook to configure your web server

Configure your web server machine for ssh access from the Chef Workstation.

  1. Authenticate the Chef Workstation machine to use Compute Engine commands.

    # gce: chef-workstation
    gcloud auth activate-service-account chef-workstation@$PROJECT_NAME.iam.gserviceaccount.com --key-file=key.json
    
  2. Connect to the web-server. This will force the creation of a new SSH key and upload it to the Compute Engine instance.

    # gce: chef-workstation
    gcloud compute ssh web-server --command='exit'
    
  3. Find the IP address of the web server.

    gcloud compute addresses list
    
  4. Use the chef-run command to run the lamp cookbook on your web server.

    # gce: chef-workstation
    cd ~/chef-repo/cookbooks
    chef-run -i ~/.ssh/google_compute_engine $USER@$WEB_SERVER_IP lamp
    
  5. Verify that the cookbook completed by retrieving the content hosted by the web server.

    # gce: chef-workstation
    curl $WEB_SERVER_IP
    

Or visit $WEB_SERVER_IP in your browser. If everything is working it will display a simple HTML table view of your Cloud SQL database.

Cleaning up

After you've finished the tutorial, you can clean up the resources you created on Google Cloud Platform so you won't be billed for them in the future. The following sections describe how to delete or turn off these resources.

Deleting the project

The easiest way to eliminate billing is to delete the project you created for the tutorial.

To delete the project:

  1. In the Cloud Platform Console, go to the Projects page.
  2. Click the trash can icon to the right of the project name.

Warning: Deleting a project has the following consequences:

If you used an existing project, you'll also delete any other work you've done in the project. You can't reuse the project ID of a deleted project. If you created a custom project ID that you plan to use in the future, you should delete the resources inside the project instead. This ensures that URLs that use the project ID, such as an appspot.com URL, remain available.

Deleting instances

To delete a Compute Engine instance:

  1. In the Cloud Platform Console, go to the VM Instances page.
  2. Click the checkbox next to your chef-workstation and web-server instances.
  3. Click the Delete button at the top of the page to delete the instances.

Deleting VPC network

To delete the VPC network you must first delete firewall rules that depend on it:

  1. In the Cloud Platform Console, go to the Firewall Rules page.
  2. Click the checkbox next to the firewall rules for the web-server-network network.
  3. Click the Delete button at the top of the page to delete the firewall rules.
  4. In the Cloud Platform Console, go to the VPC Networks page.
  5. Click on web-server-network.
  6. Click the Delete VPC Network button at the top of the page.

Delete Cloud SQL instance

  1. In the Cloud Platform Console, go to the Cloud SQL Instances page.
  2. Click the checkbox next to the SQL instance.
  3. Click the Delete button at the top of the page to delete the instance.

Delete service accounts

  1. In the Cloud Platform Console, go to the Service accounts page.
  2. Click the checkbox next to the sql and compute accounts.
  3. Click the Delete button at the top of the page to delete the service accounts.

Submit a Tutorial

Share step-by-step guides

SUBMIT A TUTORIAL

Request a Tutorial

Ask for community help

SUBMIT A REQUEST

GCP Tutorials

Tutorials published by GCP

VIEW TUTORIALS

Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License. For details, see our Site Policies. Java is a registered trademark of Oracle and/or its affiliates.