Managing applications with Application Delivery


This page walks you through configuring a deployment of NGINX with Application Delivery. The deployment runs on two environments: staging and prod. The prod environment uses regular configuration, while staging uses a slightly modified one.

Requirements

To complete this tutorial, you will need the following:

  • Git 2.19.2 or higher installed locally.
  • A GitHub or GitLab account with permissions to create a private repository. Application Delivery supports only GitHub and GitLab repositories.
  • A cluster running GKE version 1.15 or later.
  • A user with clusterAdmin privileges.
  • Kustomize installed locally. You can follow the installation guide.
  • If you want to validate your Kubernetes configuration files in the deployment repository, you must install Docker.

Before you begin

Before you start, make sure you have performed the following tasks:

Set up default gcloud settings using one of the following methods:

  • Using gcloud init, if you want to be walked through setting defaults.
  • Using gcloud config, to individually set your project ID, zone, and region.

Using gcloud init

If you receive the error One of [--zone, --region] must be supplied: Please specify location, complete this section.

  1. Run gcloud init and follow the directions:

    gcloud init

    If you are using SSH on a remote server, use the --console-only flag to prevent the command from launching a browser:

    gcloud init --console-only
  2. Follow the instructions to authorize gcloud to use your Google Cloud account.
  3. Create a new configuration or select an existing one.
  4. Choose a Google Cloud project.
  5. Choose a default Compute Engine zone for zonal clusters or a region for regional or Autopilot clusters.

Using gcloud config

  • Set your default project ID:
    gcloud config set project PROJECT_ID
  • If you are working with zonal clusters, set your default compute zone:
    gcloud config set compute/zone COMPUTE_ZONE
  • If you are working with Autopilot or regional clusters, set your default compute region:
    gcloud config set compute/region COMPUTE_REGION
  • Update gcloud to the latest version:
    gcloud components update
  • Add SSH keys to your GitHub or GitLab account.
  • Test your keys with ssh:

    GitHub

    sh ssh -T git@github.com

    GitLab

    sh ssh -T git@gitlab.com

    You might be asked to confirm connection details or your key passphrase. If the connection succeeds, a message is printed on the terminal.

Setting up Application Delivery

To use Application Delivery, you must do the following:

  1. Create a new cluster with Application Delivery enabled, or enable it on an existing GKE cluster running version 1.15 or later.
  2. Install appctl, the Application Delivery command-line tool.

Create a new cluster with Application Delivery

You can create a new cluster with Application Delivery enabled using the gcloud tool or Cloud Console.

gcloud

Create a cluster:

gcloud beta container clusters create CLUSTER_NAME \
      --cluster-version CLUSTER_VERSION\
      --addons ApplicationManager

Replace the following:

  • CLUSTER_NAME: the name of your new cluster.
  • CLUSTER_VERSION: the version of your new cluster. Must be GKE 1.15 or later.

Console

  1. Go to the Google Kubernetes Engine page in Cloud Console.

    Go to Google Kubernetes Engine

  2. Click Create.

  3. In the Standard section, click Configure.

  4. Specify a Name for your cluster.

  5. Choose a Control plane version of 1.15.x or later.

  6. Configure your cluster as desired.

  7. From the navigation pane, under Cluster, click Features.

  8. Select the Enable Application Manager checkbox.

  9. Click Create.

Enabling Application Delivery on an existing cluster

You can enable Application Delivery on an existing cluster using the gcloud tool or Cloud Console.

gcloud

To enable Application Delivery on an existing cluster, run the following command:

gcloud beta container clusters update CLUSTER_NAME \
      --update-addons ApplicationManager=ENABLED

Replace CLUSTER_NAME with the name of your existing cluster.

Console

  1. Go to the Google Kubernetes Engine page in Cloud Console.

    Go to Google Kubernetes Engine

  2. In the cluster list, click the name of the cluster you want to modify.

  3. Under Features, next to the Application Manager field, click Edit application manager.

  4. Select the Enable Application Manager checkbox.

  5. Click Save Changes.

Confirming installation

To check the status of your Application Delivery installation, perform the following:

  1. Check the Deployment status:

    kubectl get deployment application-controller-manager -n application-system
    

    The output is similar to the following:

    NAME                             READY   UP-TO-DATE   AVAILABLE   AGE
    application-controller-manager   2/2     2            2           1h
    

    There should be two pods available in this application-controller-manager Deployment.

  2. Check the StatefulSet status:

    kubectl get statefulset kalm-controller-manager -n kalm-system
    

    The output is similar to the following:

    NAME                      READY   AGE
    kalm-controller-manager   1/1     1h
    

    There should be one Pod ready in this kalm-controller-manager StatefulSet.

Install appctl

To install the Application Delivery command-line tool appctl, use the gcloud tool to install pkg.

gcloud components install pkg

After enabling Application Delivery on a cluster and installing pkg, you are ready to deploy your first application.

Deploying an application

To deploy an application, do the following:

  1. Create new Git repositories or initialize existing repositories.
  2. Create a base configuration.
  3. Create one or more environments for your deployment.
  4. Optionally, apply configuration overlays to your environments in your application repository.
  5. Create a release candidate in the form of a pull or merge request.
  6. Deploy your release.

Creating new repositories

Create repositories for Application Delivery on GitHub or GitLab with appctl.

  1. Change to the directory where you would like to create your application directory.
  2. Create your Application Delivery repositories with appctl.

    GitHub

    Run the following command:

    appctl init APP_NAME \
        --app-config-repo=github.com/USERNAME/APP_NAME
    

    Replace the following:

    • APP_NAME: the name of your application.
    • USERNAME: your GitHub username.

    For example, if your GitHub username is octocat and you want to create an application named myapp, run the following:

    appctl init myapp \
        --app-config-repo=github.com/octocat/myapp
    

    GitLab

    Run the following command:

    appctl init APP_NAME \
        --app-config-repo=gitlab.com/USERNAME/APP_NAME
    

    Replace the following:

    • APP_NAME: the name of your application.
    • USERNAME: your GitLab username.

    For example, if your GitLab username is alice and you want to create an application named myapp, run the following:

    appctl init myapp \
        --app-config-repo=gitlab.com/alice/myapp
    
  3. appctl prompts you to confirm your new private repositories.

appctl creates two remote private Git repositories:

  • The application repository github.com/USERNAME/APP_NAME. This repository is cloned to the current directory.
  • The deployment repository github.com/USERNAME/APP_NAME-deployment. The local deployment repository is stored in ./APP_NAME/.deployment.

For more information on the content and structure of these repositories, see the Application delivery concept guide.

Initialize existing repositories

If you have existing repositories, you can initialize the repositories for Application Delivery on GitHub or GitLab with appctl.

  1. Change to the directory where you want to create your application directory.

  2. Run the appctl init command, which creates a directory named APP_NAME and clones your repository there.

    The appctl init command initializes a Kustomize base layer in the configuration files stored in the ./config directory of the repository. You can specify a different configuration path with the --config-path flag.

    By default, appctl init uses github.com/USERNAME/APP_NAME-deployment as the URL of the deployment repository. You can use the --deployment-repo flag to specify a different URL. If the deployment repository doesn't exist, the appctl command will create one.

    GitHub

    appctl init APP_NAME \
          --app-config-repo=github.com/USERNAME/APP_NAME \
          [--config-path=CONFIG_PATH]
    

    Replace the following:

    • APP_NAME: the name of your application.
    • USERNAME: your GitHub username.
    • CONFIG_PATH: the optional path to the configuration directory of your repository. If omitted, the default is ./config.

    For example, if the existing application config repository is https://github.com/octocat/myapp and you expect the deployment repository to be https://github.com/octocat/myapp-deploy, run the following:

    appctl init myapp \
        --app-config-repo=github.com/octocat/myapp
    

    GitLab

    appctl init APP_NAME \
        --app-config-repo=gitlab.com/USERNAME/APP_NAME \
        [--config-path=CONFIG_PATH]
    

    Replace the following:

    • APP_NAME: the name of your application.
    • USERNAME: your GitLab username.
    • CONFIG_PATH: the optional path to the configuration directory of your repository. If omitted, the default is ./config.

    For example, if the existing application config repository is gitlab.com/alice/myapp, run the following:

    appctl init myapp --app-config-repo=gitlab.com/alice/myapp
    

Creating a base configuration

  1. Change your working directory to your application repository.

  2. Create the configuration for your Kubernetes workload. This can be any valid Kubernetes deployment.

    The following configuration defines an application named nginx, which deploys 3 replicas of the nginx container. Copy the configuration into the file config/base/myapp.yaml . If you would like to enable a LoadBalancer, uncomment the line type: LoadBalancer.

    #myapp/config/base/myapp.yaml
    
    apiVersion: v1
    kind: Service
    metadata:
      name: nginx
      labels:
        app: nginx
    spec:
      # if your cluster supports it, uncomment the following to automatically create
      # an external load-balanced IP for the frontend service.
      # type: LoadBalancer
      ports:
        - port: 80
      selector:
        app: nginx
    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: nginx
    spec:
      replicas: 3
      selector:
        matchLabels:
          app: nginx
      template:
        metadata:
          labels:
            app: nginx
        spec:
          containers:
          - name: nginx
            image: nginx:1.7.9
            ports:
            - containerPort: 80
    
  3. Configure Application Delivery to apply this configuration to the base. Paste the following into config/base/kustomization.yaml.

    #config/base/kustomization.yaml
    
    resources:
      - myapp.yaml
    

Testing and pushing your configuration

  1. From your application repository directory, test your configuration with kustomize build:

    kustomize build config/base/
    

    If your configuration is valid, kustomize prints the YAML that will be deployed to your cluster when it is applied.

  2. After you have validated your YAML, create and push a commit in your application repository:

    git add .
    git commit -m "Creating APP_NAME"
    git push origin master
    

Adding a release environment

Application Delivery deploys your application into environments. You add environments for your releases with appctl.

  1. Change to your application repository root directory.

  2. Create your environment with appctl:

    appctl env add ENVIRONMENT_NAME \
        --cluster=CLUSTER_NAME
    

    Replace the following:

    • ENVIRONMENT_NAME: the name of your new environment.
    • CLUSTER_NAME: the name of your cluster.

    appctl creates a git commit containing a scaffolded Kustomize configuration.

    You can specify the name of the namespace for a release environment of this application using --namespace. If not, the default namespace name is APP_NAME-ENVIRONMENT_NAME.

    For example, to add the staging and prod environments to the cluster application-cluster using the default namespace name, run the following command:

    appctl env add staging --cluster=application-cluster
    appctl env add prod --cluster=application-cluster
    

    To add the test environment in the test namespace to the cluster application-cluster, run the following command:

    appctl env add test --cluster=application-cluster --namespace=test
    

    When adding an environment, you need to consider if pull requests and code reviews are needed in the deployment repository for this environment. By default, pull requests are created. If the environment is not production critical and code review is not necessary, you can skip creating pull requests with --review-required=false.

    For example, to add the test environment, which doesn't require pull requests, run the following command:

    appctl env add test \
        --cluster=application-cluster \
        --review-required=false
    
  3. Optionally, view the changes Application Delivery made in your Git repository with git log:

    git log -p *
    
  4. Push the configuration to your application repository:

    git push origin master
    

Optional: Checking your deployment repository

Open the GitHub or GitLab page for your deployment repository. For example, if your GitHub username is octocat and you created an application named myapp, the URL is https://github.com/octocat/myapp-deployment. From this page, you can see the branches that were created for each environment.

Deploying an environment

To deploy an environment with Application Delivery, do the following:

  1. Create a version with git tag and push that tag:

    git tag VERSION
    git push origin VERSION
    

    Replace VERSION with the version number for your application.

    For example, to push version v0.1.0, run the following commands:

    git tag v0.1.0
    git push origin v0.1.0
    ```
    
  2. Use appctl prepare to nominate the currently tagged version and generate a pull request in the deployment repository for review.

    appctl prepare ENVIRONMENT_NAME
    

    For example, to use the staging environment, run the following command:

    appctl prepare staging \
        --validate=true
    

    This command triggers the kpt validation function gcr.io/kustomize-functions/example-validator to validate the change being pushed to the deployment repository.

    kpt is installed by gcloud components install pkg. This validation function runs kubeval under the hood, which validates Kubernetes configuration files using schemas from the Kubernetes OpenAPI spec.

    To run kpt prepare staging --validate, you must install Docker on your machine.

    By default, the --validate flag is disabled.

    If appctl completed the commit to the deployment repository, it prints a URL to a pull request, like the following:

    Created a "Pull Request": "https://github.com/octocat/myapp-deployment/pull/[Pull_Request_ID]"
    
  3. Use GitHub or GitLab to review and approve the pull request.

  4. After the pull request has been approved, use appctl apply to complete the deployment:

    appctl apply ENVIRONMENT_NAME
    

    Replace ENVIRONMENT_NAME with the name of your environment.

    For example, to deploy changes to the staging environment, run the following command:

    appctl apply staging
    
  5. Confirm that your application is running with kubectl or from Cloud Console.

    kubectl

    Use kubectl describe to view the status of your application:

    kubectl get releasetracks.app.gke.io APP_NAME \
        --n NAMESPACE \
        -w
    

    Replace the following:

    • APP_NAME: the name of your application repository.
    • NAMESPACE: the namespace name you specified when creating the environment. If you didn't specify a namespace name, the default is APP_NAME-ENVIRONMENT_NAME.

    For example, to check the status of the staging environment of the application named myapp, run the following command:

    kubectl get releasetracks.app.gke.io myapp -n myapp-staging
    

    To check the status of the test environment which was added earlier by env add --namespace test, run the following command:

    kubectl get releasetracks.app.gke.io myapp -n test
    

    Console

    To see status and version information for your applications deployed with Application Delivery see the GKE Applications page on Cloud Console.

Promoting a release

  1. To promote a release candidate from one environment to another, run the following command:

    appctl prepare TARGET_ENVIRONMENT \
        --from-env=SOURCE_ENVIRONMENT
    

    Replace the following:

    • TARGET_ENVIRONMENT: the name of the environment to which you want to deploy the release candidate.
    • SOURCE_ENVIRONMENT: the name of the current environment.

    For example, to promote staging to prod run:

    appctl prepare prod --from-env=staging
    
  2. Use GitHub or GitLab to review and approve the pull request.

  3. To deploy the release candidate to the target environment, run the following command:

    appctl apply TARGET_ENVIRONMENT
    

    For example, to deploy to the prod environment, run the following command:

    appctl apply prod
    

Viewing environments in Cloud Console

After the application of an environment is deployed, you can view it on GKE Applications page. This page contains a list of applications with their environments marked inside parentheses. Here is a screenshot of an application myapp with two environments staging and prod. The namespace, cluster and version of each environment are also displayed on this page.

A screenshot of the application page.

To see details of the application, such as the git tag, components, and environments list, click the application name. The following screenshot shows the details for myapp.

A screenshot of the application detail page.

Changing an environment's configuration

This section assumes you have a staging environment configured as in the previous steps. You might need to adapt these instructions for your use.

In this section, you change the parameters for the staging environment using a kustomize overlay. After making the change, you push and tag your changes in git. Application Delivery will update your cluster.

  1. Create a file named config/envs/staging/patch-replicas.yaml, and copy the following text into it. This updates the configuration in the staging environment to run one replica instead of three replicas.

    #config/envs/staging/patch-replicas.yaml
    
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: nginx
    spec:
      replicas: 1
    
  2. Edit the config/envs/staging/kustomization.yaml file and add patch-replicas.yaml to a new collection named patchesStrategicMerge.

    #config/envs/staging/kustomization.yaml
    namespace: myapp-staging
    bases:
       - ../../base
    patchesStrategicMerge:
       - patch-replicas.yaml
    

    You can also add environment-specific annotations in this overlay. The following example adds an annotation named oncall-team to add all resources under this environment. For more information, see Kustomize file fields.

    #config/envs/staging/kustomization.yaml
    
    #Don't change the namespace field
    namespace: myapp-staging
    bases:
       - ../../base
    patchesStrategicMerge:
       - patch-replicas.yaml
    commonAnnotations:
      oncall-team: staging-oncall@foo.bar
    
  3. Test your configuration with kustomize build:

    kustomize build config/envs/staging/
    
  4. Add and commit your changes.

    git add .
    git commit -m "<var>COMMIT_MESSAGE</var>"
    git push origin master
    

    Replace COMMIT_MESSAGE with a message that describes your changes.

  5. Create a version with git tag and push it.

    git tag VERSION
    git push origin VERSION
    

    Replace VERSION with the version number you want.

  6. Use appctl prepare to generate a pull request in the deployment repository for review:

    appctl prepare ENVIRONMENT_NAME
    
  7. Follow the link to create a GitHub or GitLab pull request.

  8. Look over the contents of your pull request. Application Delivery makes a one-line change that sets the value of replicas to 1.

  9. Approve the pull request with GitHub or GitLab.

  10. Use appctl apply to apply the changes:

    appctl apply staging
    

Rolling back configuration changes

To roll back to a previous release, run the following command:

appctl apply TARGET_ENVIRONMENT \
    --from-tag GIT_TAG

Replace the following:

  • TARGET_ENVIRONMENT: the name of the environment in which you want to deploy the release.
  • GIT_TAG: the name of the tag for the release.

Using appctl in scripts

The appctl tool is interactive and expects user input by default. If you want to run appctl in a script, container, or pipelines, set the environment variable APPCTL_INTERACTIVE to false.

For example, in the bash shell, run the following command:

export APPCTL_INTERACTIVE=false

Information on specific appctl commands is available with appctl help command. For example, to get help with appctl prepare, run appctl help prepare.

Uninstalling Application Manager

To remove the application running in your cluster, you delete all namespaces created with new environments.

For all of your environments and clusters, repeat the following commands:

  1. Switch to the cluster for a given environment:

    kubectl config use-context ENVIRONMENT_CLUSTER_NAME
    

    Replace ENVIRONMENT_CLUSTER_NAME with the name of the cluster in the selected environment.

  2. Delete the namespace where your application for this environment is running:

    kubectl delete ns NAMESPACE
    

    Replace NAMESPACE with the namespace name you want to delete. The default is APP_NAME-ENVIRONMENT_NAME.

  3. From GitHub or GitLab, delete the two Git repositories created by appctl.

  4. Delete your local application directory:

    rm -rf myapp/
    
  5. You can disable Application Delivery in your cluster from gcloud or Cloud Console:

    gcloud

    gcloud beta container clusters update CLUSTER_NAME \
        --update-addons ApplicationManager=DISABLED
    

    Console

    1. Go to the Google Kubernetes Engine page in Cloud Console.

      Go to Google Kubernetes Engine

    2. In the cluster list, click the name of the cluster you want to modify.

    3. Under Features, next to the Application Manager field, click Edit application manager.

    4. Clear the Enable Application Manager checkbox.

    5. Click Save Changes.

What's next

Read more about Kustomize.