Automated Network Deployment: Startup

This tutorial outlines a simple deployment of a single virtual machine (VM) instance in the default Virtual Private Cloud (VPC) network. This tutorial is the second in a three-part series that demonstrates the automated deployment of common networking resource patterns using Cloud Deployment Manager and Terraform by HashiCorp.

To use this tutorial, you must first set up your environment with proper authentication by working through the Automated Network Deployment: Overview tutorial.

Objectives

Prepare a simple working environment in Google Cloud Platform (GCP).
Understand the initial configuration files for a simple deployment.
Run common commands in Deployment Manager to deploy a VM instance.
Run common commands in Terraform to deploy a VM instance.

Costs

This tutorial uses the following billable components of Google Cloud:

Compute Engine VM instance
Persistent disk
Network egress

To generate a cost estimate based on your projected usage, use the pricing calculator. New Google Cloud users might be eligible for a free trial.

Before you begin

Select the GCP project named gcp-automated-networks.

Go to the Projects Page
Start a Cloud Shell instance. You run all the terminal commands in this tutorial from Cloud Shell.

Open Cloud Shell

Deployment architecture

In this tutorial, you build the following deployment environment:

deployment environment

The first step is to create a VM instance in the default VPC network. You check that it's up and running using the ping command, and you access the instance using the Secure Shell (ssh) tool.

After you create your deployment, you navigate to the Cloud Console VPC networks panel. Notice that a default VPC network has been automatically created for your project. This VPC contains two premade resource collections:

Firewalls

Defines the basic firewall rules, allowing HTTP(S) and ping traffic.
Routes

Creates routes, allowing traffic to flow between various regions and the default internet gateway.

New users find that these network resource collections make initial use of GCP easier.

Preparing your GCP working environment

This tutorial relies on the project configuration and tutorial code cloned in the Overview tutorial.

Examining configuration files

Configuration files for Deployment Manager

In Deployment Manager, a deployment configuration is defined by an overall YAML configuration file and a set of related resource template files. In this tutorial, a minimum set of files has been provided to illustrate the concepts. This is a functional deployment and requires no edits to run.

Filename	Purpose
`autonetdeploy_config.yaml`	Imports the `autonetdeploy-startup.py` configuration template. Sets values for the following required deployment variables: `project_id`, `zone`, `instance_type`.
`autonetdeploy_startup.py.schema`	Defines the required deployment variables for the configuration template. Review this file to see how to use the template without needing to understand every underlying detail of the deployment configuration.
`autonetdeploy_startup.py`	Contains the detailed resource definitions for the deployment configuration (for example, `compute.v1.instance`). This can be a YAML file, but Python allows greater flexibility.

Configuration files for Terraform

In Terraform, a deployment configuration is defined by a directory of files. Although these files can be JSON files, it's better to use the Terraform configuration file (.tf file) syntax, which is easier to read and maintain. This tutorial provides a set of files that illustrate one way of cleanly organizing your resources. This set is a functional deployment and requires no edits to run.

Filename	Purpose
`main.tf`	Defines your providers and specifies which clouds to deploy in this configuration. Also reads your credentials, project, and desired region.
`gcp_variables.tf`	Declares variables used to parameterize and customize the deployment—for example, `gcp_region` and `gcp_instance_type`.
`gcp_compute.tf`	Defines the compute resources used in your deployment—for example, `google_compute_instance`.
`gcp_outputs.tf`	Defines variables to be output or displayed upon completion of the deployment—for example, the `external_ip` and `internal_ip` of the deployed VM instance.
`terraform.tfstate`	[OUTPUT]. File used by Terraform to store the client-side resource state after checking with the cloud. For more details, see Managing GCP Projects with Terraform.
`run_graph.sh`	Shell script for generating a PNG file from Terraform that shows resource dependencies. You can see the output of this script in images/startup_plan_graph.png.

In this tutorial, you use scripts to create the terraform.tfvars file, which includes user-specific setup for credentials and gcp_project_id.

Deploying a Compute Engine instance in the default VPC network

All the automated network deployment tutorials follow the same steps to deploy and clean up a deployment architecture, as outlined in the following image.

deployment and cleanup

Deploy with Deployment Manager

When you create a deployment, whether in preview mode or not, Deployment Manager ingests the resource configuration files, assesses resource dependencies, and prepares to build the environment as efficiently as possible. Each deployment is tagged with a unique deployment_name that you can use later to describe, update, or destroy the deployment.

In Cloud Shell, navigate to the deploymentmanager directory:
```
pushd deploymentmanager
```

Preview the deployment using the --preview flag. This creates a deployment on GCP without using any resources.

gcloud deployment-manager deployments create defaultvm-deployment --config ./autonetdeploy_config.yaml --preview

Next, inspect what you created to be certain it's what you expect:

gcloud deployment-manager deployments list

Resulting output:

NAME                  LAST_OPERATION_TYPE  STATUS  DESCRIPTION  MANIFEST  ERRORS
defaultvm-deployment  preview              DONE                           []

Now check how your deployments are described:

gcloud deployment-manager deployments describe defaultvm-deployment

Resulting output:

...
NAME                 TYPE                 STATE       INTENT
gcp-disk-us-west1-b  compute.v1.disk      IN_PREVIEW  CREATE_OR_ACQUIRE
gcp-vm-us-west1-b    compute.v1.instance  IN_PREVIEW  CREATE_OR_ACQUIRE

Because the preview deployment is just documentation for a planned deployment, go ahead and delete the preview deployment.

gcloud deployment-manager deployments delete defaultvm-deployment

Resulting output:

...
Do you want to continue (y/N)?  y
...
Delete operation operation-[id] completed successfully.

Create a real deployment. Use the following command to deploy a compute.v1.disk resource and a compute.v1.instance resource, usually within 30 seconds:

gcloud deployment-manager deployments create defaultvm-deployment --config ./autonetdeploy_config.yaml

Resulting output:

...
NAME                 TYPE                 STATE      ERRORS  INTENT
gcp-disk-us-west1-b  compute.v1.disk      COMPLETED  []
gcp-vm-us-west1-b    compute.v1.instance  COMPLETED  []

Make sure that you created a valid deployment:

gcloud deployment-manager deployments list

Resulting output:

NAME                  LAST_OPERATION_TYPE  STATUS  DESCRIPTION  MANIFEST       ERRORS
defaultvm-deployment  insert               DONE                 manifest-[ID]  []

Inspect the deployed resources to verify their details:

gcloud deployment-manager deployments describe defaultvm-deployment

Resulting output:

fingerprint: ...
id: ...
insertTime: ...
manifest: ...
name: defaultvm-deployment
operation:
  endTime: ...
  name: ...
  operationType: insert
  progress: 100
  startTime: ...
  status: DONE
  user: ...
NAME                 TYPE                 STATE      INTENT
gcp-disk-us-west1-b  compute.v1.disk      COMPLETED
gcp-vm-us-west1-b    compute.v1.instance  COMPLETED

Verify that your deployment has successfully created running instances:
```
gcloud compute instances list
```
Resulting output (scroll right to see all of it):
```
NAME               ZONE        MACHINE_TYPE   PREEMPTIBLE  INTERNAL_IP  EXTERNAL_IP        STATUS
gcp-vm-us-west1-b  us-west1-b  n1-standard-1               10.138.0.2   [GCP_EXTERNAL_IP]  RUNNING
```
In the Cloud Console, you can review and verify these instances in the Deployment Manager or VM instances panel. In this deployment, one instance is deployed in the default VPC network.
To communicate with your running instance, use the ssh command to connect to the VM, and then you can run commands, such as ping and curl. Recall that you registered the required vm-ssh-key as project-wide metadata in the Automated Network Deployment: Overview tutorial.
```
ssh -i ~/.ssh/vm-ssh-key [GCP_EXTERNAL_IP]
```
Run the following commands in your ssh session:
```
ping -c 5 google.com
curl ifconfig.co/ip
```
After you verify that everything is working properly, enter the following command:
```
exit
```

View the manifest file, which you might find useful when debugging your deployments. The file consists of read-only information that describes the deployed resources in detail, along with any errors or warnings. If you are not sure what the manifest file is named, look for it in the output of the manifests list command. The manifest filename is of the form: manifest-###.

gcloud deployment-manager manifests list --deployment defaultvm-deployment

Resulting output:

NAME                    ID                   INSERT_TIME
manifest-1234567890000  3456789012394315470  2017-08-22T...

gcloud deployment-manager manifests describe [MANIFEST_FILENAME] --deployment defaultvm-deployment

Resulting output (scroll right to see all of it):

...
name: [MANIFEST_FILENAME]
selfLink: https://www.googleapis.com/deploymentmanager/v2/projects/gcp-automated-networks/global/deployments/defaultvm-deployment/manifests/[MANIFEST_FILENAME]

Clean up with Deployment Manager

Clean up the deployed resources. You will continue to be billed for your VM instance until you run the delete deployment command. Because delete permanently deletes your resources, confirm your intentions by entering y for yes. This delete command usually finishes within 30 seconds.

gcloud deployment-manager deployments delete defaultvm-deployment

Resulting output:

The following deployments will be deleted:
- defaultvm-deployment

Do you want to continue (y/N)?  y

gcloud deployment-manager deployments list

Resulting output:

Listed 0 items.

popd > /dev/null

You have now successfully deployed and deleted a VM instance in GCP using Deployment Manager.

Deploy with Terraform

Deployments in Terraform are a little different. Terraform uses a file, terraform.tfstate, to capture the resource state. To view the current state in a readable form, you run terraform show.

In Cloud Shell, navigate to the terraform directory.
```
pushd terraform
```
You might need to re-export your PATH to the ~/terraformdirectory. You can re-run the following script from Cloud Shell to see the export command.
```
./get_terraform.sh
```
Validate the syntax of your configuration files. Terraform provides the validate command to check for valid syntax across all your configuration files.
```
terraform validate
```
This validation check is simpler than those performed as part of plan and apply in subsequent steps. The validate command does not authenticate with any providers.

If you don't see an error message, you have completed an initial validation of your file syntax and basic semantics. If you do see an error message, the validation failed.
Preview the deployment. The Terraform plan command previews a deployment without instantiating resources in the cloud:
```
terraform plan
```
The plan command differs from the Deployment Manager preview command in that it does not create a deployment reference, in the cloud, that must be deleted. Also, the plan command requires successful authentication with all providers indicated in the configuration.

The plan command returns an output listing of resources to be added, removed, or updated. The last line of the plan output shows a count of resources to be added, changed, or destroyed.

Resulting output:
```
Refreshing Terraform state in-memory prior to plan...
...
Plan: 1 to add, 0 to change, 0 to destroy.
```
Optionally, visualize your resource dependencies by using the Terraform graph command. The dependency graph helps you analyze your deployed resources. You can view a previously prepared version of the output file at images/startup_plan_graph.png.

The run_graph.sh script creates the PNG file ./startup_plan_graph.png, which looks similar to this:

The run_graph.sh script relies on the graphviz package. If graphviz is not installed, you see the message dot: command not found. In Cloud Shell, install graphviz by using the following command:
```
sudo apt-get install graphviz
```
Create a deployment:
```
terraform apply
```
The apply command creates a deployment with backing resources in the cloud. In a little over ten seconds, your google_compute_instance resource is created, including a boot disk and a network interface with both internal and external IP addresses.

The apply command output includes details of the resources deployed and the output variables that the configuration defines:
```
data.google_compute_zones.available: Refreshing state...
...
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
...
Outputs:

gcp_instance_external_ip = [GCP_EXTERNAL_IP]
gcp_instance_internal_ip = [GCP_INTERNAL_IP]
```
If you need to update the expected end state of your configuration, edit your .tf files. Add a startup script to your google_compute_instance by editing gcp_compute.tf:
```
zone = "${data.google_compute_zones.available.names[0]}"
metadata_startup_script = "echo hi > /test.txt"
```
Terraform analyzes the edits and identifies the minimum changes needed to update your deployment state to match. Apply your changes:
```
terraform apply
```
Terraform destroys gcp-vm and re-creates it with the updated configuration.
Your deployments can emit output variables to aid your workflow. In this tutorial, gcp_outputs.tf identifies the assigned internal and external IP addresses as output variables. The addresses are printed automatically when the apply step completes. If, later in your workflow, you want to redisplay the output variable values, use the output command:
```
terraform output
```
The output variables that this configuration defines include the internal and external IP addresses for your GCP instance. When you use the ssh command, you need the external GCP IP address to connect to the instance.
```
gcp_instance_external_ip = [GCP_EXTERNAL_IP]
gcp_instance_internal_ip = [GCP_INTERNAL_IP]
```

Inspect the deployed resources to verify the current state:

terraform show

Resulting output:

data.google_compute_zones.available:
...
google_compute_instance.gcp-vm:
...

Review instances by using gcloud compute instances list or by using the Cloud Console, on the Deployment Manager or VM instances panel. For this deployment, this one instance is deployed in the default VPC network.
```
gcloud compute instances list
```
Resulting output:
```
NAME             ZONE        MACHINE_TYPE   PREEMPTIBLE  INTERNAL_IP  EXTERNAL_IP
gcp-vm-us-west1  us-west1-a  n1-standard-1               10.138.0.2   [GCP_EXTERNAL_IP]  RUNNING
```
Verify that your VM instance is functioning by running the following ssh command:
```
ssh -i ~/.ssh/vm-ssh-key [GCP_EXTERNAL_IP]
```

Run the following commands in your ssh session:

ping -c 5 google.com
curl ifconfig.co/ip`
exit

Resulting output:

Connection to [GCP_EXTERNAL_IP] closed.

Clean up with Terraform

Clean up the deployed resources. You will continue to be billed for your VM instance until you run the destroy deployment command.

Check what will happen without actually destroying any resources.
```
terraform plan -destroy
```
Resulting output:
```
...
Plan: 0 to add, 0 to change, 1 to destroy.
```

Permanently destroy the deployed resources in the cloud. Confirm your intentions by entering yes. This destroy command usually completes in about 30 seconds.

terraform destroy

Resulting output:

Do you really want to destroy?
Terraform will delete all your managed infrastructure.
There is no undo. Only 'yes' will be accepted to confirm.
Enter a value: yes

...
Destroy complete! Resources: 1 destroyed.

Verify that the resources are no longer deployed.
```
terraform show
```
Because you just destroyed all resources, the show command displays no lines, indicating that no resources remain deployed.
Finish by running the following command:
```
popd > /dev/null
```

You have now successfully used Terraform to deploy and destroy a VM instance in GCP.

What's next

Start the Building a VPN between GCP and AWS tutorial, which walks you through constructing VM instances in custom VPC networks in GCP and Amazon Web Services (AWS). After you create the VM instances, you deploy a virtual private network (VPN) between the associated VPC networks. Then, you can run connectivity tests using private IPs between the VM instances in GCP and AWS.
Learn about a more advanced method of storing your Terraform state file (terraform.tfstate) in Cloud Storage: Managing GCP Projects with Terraform.
Try out other Google Cloud features for yourself. Have a look at our tutorials.