Building Windows Server multi-arch images


This tutorial demonstrates how to manage the versioning complexity of building images that target multiple Windows Server versions. Windows Server containers have version compatibility requirements that prevent containers from running on more than one Windows Server host version. However, Docker on Windows Server supports multi-arch (or multi-platform) container images that can run across multiple Windows Server versions.

With multi-arch images, you can upgrade your Google Kubernetes Engine (GKE) Windows Server node pools to your preferred Windows Server version without rebuilding the image and changing the Pod specs. For example:

  • GKE version 1.15 supports Windows Server 1809
  • GKE version 1.16 supports Windows Server 1909

To upgrade from one GKE version to a later version automatically, you must build multi-arch images for your Windows workloads. Building a multi-arch image involves building an image for each Windows Server version, and then building a manifest that references those images for each Windows Server version. You can build the images manually if you want full control over the image creation and build process. Alternatively, you can use Cloud Build to automatically build the Windows Server multi-arch images.

Objectives

In this tutorial, you learn how to create Windows Server multi-arch images manually or by using Cloud Build.

  • Build the images manually:

    • Create 2 Docker images with different versions or types of Windows Server, for example Long-Term Servicing Channel (LTSC) and Semi-Annual Channel (SAC).
    • Create a Windows Server VM.
    • Create a manifest and push it to the registry.
  • Build the images using Cloud Build:

    • Prepare your environment by creating a project, enabling APIs, and granting permissions.
    • Create an application, Dockerfiles, and build files.
    • Run a command to build the image.

Costs

This tutorial uses billable components of Google Cloud, including:

  • Compute Engine
  • Cloud Build

Use the pricing calculator to generate a cost estimate based on your projected usage. New Google Cloud users might be eligible for a free trial.

Before you begin

Before you start, make sure that you have performed the following tasks:

  1. Install the Cloud SDK to run gcloud commands.
  2. Install Docker to build containers.
  3. Install Go to build Windows Server binaries.

Building multi-arch images manually

Building multi-arch images manually provides you with the flexibility to build an image that includes any Windows Server versions that you need. To build a multi-arch image manually:

  1. Create an LTSC 2019 Docker single-arch image. See details about creating Docker images in Deploying a Windows Server application. For example, gcr.io/my-project/foo:1.0-2019.
  2. Create a SAC 1909 Docker single-arch image. For example, gcr.io/my-project/foo:1.0-1909.
  3. Create a Windows Server VM, for example version 1909. See the Quickstart using a Windows Server VM.
  4. Use RDP to connect to the VM.
  5. Open a PowerShell window to run the commands in the next steps.
  6. Enable the docker manifest experimental feature. A Docker manifest is a list of images to push to a registry:

    PS C:\> $env:DOCKER_CLI_EXPERIMENTAL = 'enabled'
    
  7. Create the multi-arch manifest:

    docker manifest create gcr.io/my-project/foo:1.0 gcr.io/my-project/foo:1.0-2019 gcr.io/my-project/foo:1.0-1909
    
  8. Push the newly created multi-arch image manifest to your Container Registry:

     docker manifest push gcr.io/my-project/foo:1.0
    
  9. To ensure that your multi-arch image was built and pushed successfully, navigate to gcr.io/my-project/foo and click on that image. You'll see the 3 images inside:

    • foo:1.0-2019
    • foo:1.0-1909
    • foo:1.0

Now you can refer to the multi-arch image gcr.io/my-project/foo:1.0 in your Pod specs. This will let you safely use auto-upgrade for your GKE Windows node pools.

Building multi-arch images using the Cloud Build gke-windows-builder

To ease the effort of the manual build steps, you can use the gke-windows-builder based on the OSS windows-builder. You can use the gke-windows-builder with Cloud Build to build the Windows Server multi-arch images automatically. GKE updates the builder to include new supported Windows SAC and LTSC versions when they are released. Another benefit of using the builder is that you don't have to create your own Windows VM with Powershell to build the images. The Windows VM is replaced by a Docker container that runs the commands for you inside Cloud Build.

To help you understand how the builder works, follow this example to build a "hello world" multi-arch image. These steps can be performed on Linux or Windows servers.

Preparing the environment

To prepare your environment, complete the following steps:

  1. Create a workspace directory on your work machine, for example: ~/gke-windows-builder/hello-world.
  2. Create or select a project for this tutorial.
  3. Make sure that billing is enabled for your project.
  4. Make sure that the Cloud Build API is enabled for your project.

    Enable the Cloud Build API

  5. Grant the following Identity and Access Management (IAM) roles to your Cloud Build service account by using the gcloud command-line tool:

    1. Enable the API to allow gke-windows-builder to create and manage Windows Server VMs:

      gcloud services enable compute.googleapis.com
      
    2. Set variables:

      export PROJECT=$(gcloud info --format='value(config.project)')
      export MEMBER=$(gcloud projects describe $PROJECT --format 'value(projectNumber)')@cloudbuild.gserviceaccount.com
      
    3. Assign roles. These roles are required for the builder to create the Windows Server VMs, to copy the workspace to a Cloud Storage bucket and to configure the networks to build the Docker image:

      gcloud projects add-iam-policy-binding $PROJECT --member=serviceAccount:$MEMBER --role='roles/compute.instanceAdmin'
      gcloud projects add-iam-policy-binding $PROJECT --member=serviceAccount:$MEMBER --role='roles/iam.serviceAccountUser'
      gcloud projects add-iam-policy-binding $PROJECT --member=serviceAccount:$MEMBER --role='roles/compute.networkViewer'
      gcloud projects add-iam-policy-binding $PROJECT --member=serviceAccount:$MEMBER --role='roles/storage.admin'
      
  6. Add a firewall rule named allow-winrm-ingress to allow WinRM to connect to Windows Server VMs to run a Docker build:

    gcloud compute firewall-rules create allow-winrm-ingress --allow=tcp:5986 --direction=INGRESS
    

Creating the hello.exe binary in your workspace

For this tutorial, create a simple "hello world" application, written in Go. The code for the sample app is on GitHub.

  1. Clone the repository containing the sample code for this tutorial to your local machine by using the following commands:

     git clone https://github.com/GoogleCloudPlatform/kubernetes-engine-samples
     cd kubernetes-engine-samples/windows-multi-arch
    
  2. The hello.go file prints the words "Hello World":

    // Copyright 2020 Google LLC
    //
    // Licensed under the Apache License, Version 2.0 (the "License");
    // you may not use this file except in compliance with the License.
    // You may obtain a copy of the License at
    //
    //     http://www.apache.org/licenses/LICENSE-2.0
    //
    // Unless required by applicable law or agreed to in writing, software
    // distributed under the License is distributed on an "AS IS" BASIS,
    // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    // See the License for the specific language governing permissions and
    // limitations under the License.
    
    package main
    import "fmt"
    func main() {
        fmt.Println("Hello World!")
    }
  3. Generate the hello.exe binary:

    GOOS=windows go build hello.go
    

You'll see the hello.exe binary in your workspace.

Creating a Dockerfile and build files in your workspace

In this section you use a Dockerfile to build each single-arch Windows Server image and then use a build file to trigger the Cloud Build. The build combines the single-arch images into a multi-arch image.

  1. The Dockerfile is a text document that contains instructions for Docker to build an image. The gke-windows-builder replaces the WINDOWS_VERSION with a specific Windows Server version to build the image for. For example, the builder will run docker build -t multi-arch-helloworld:latest_1909 --build-arg WINDOWS_VERSION=1909 . on Windows Server 1909.

    # Copyright 2020 Google LLC
    #
    # Licensed under the Apache License, Version 2.0 (the "License");
    # you may not use this file except in compliance with the License.
    # You may obtain a copy of the License at
    #
    #     http://www.apache.org/licenses/LICENSE-2.0
    #
    # Unless required by applicable law or agreed to in writing, software
    # distributed under the License is distributed on an "AS IS" BASIS,
    # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    # See the License for the specific language governing permissions and
    # limitations under the License.
    
    ARG WINDOWS_VERSION=
    FROM mcr.microsoft.com/windows/servercore:${WINDOWS_VERSION}
    COPY hello.exe /hello.exe
    USER ContainerUser
    ENTRYPOINT ["hello.exe"]
    
  2. In the same directory that contains the Dockerfile, the cloudbuild.yaml file is your build config file. At build time, Cloud Build automatically replaces $PROJECT_ID with your project ID.

    # Copyright 2020 Google LLC
    #
    # Licensed under the Apache License, Version 2.0 (the "License");
    # you may not use this file except in compliance with the License.
    # You may obtain a copy of the License at
    #
    #     http://www.apache.org/licenses/LICENSE-2.0
    #
    # Unless required by applicable law or agreed to in writing, software
    # distributed under the License is distributed on an "AS IS" BASIS,
    # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    # See the License for the specific language governing permissions and
    # limitations under the License.
    
    timeout: 3600s
    steps:
    - name: 'gcr.io/gke-release/gke-windows-builder:release-2.5.0-gke.0'
      args:
      - --container-image-name
      - 'gcr.io/$PROJECT_ID/multiarch-helloworld:latest'
    

Building the image

Now you can build the image and view your logs to verify a successful build.

  1. To build the image, run the following command:

    gcloud builds submit --config=cloudbuild.yaml .
    
  2. You'll see logs like the following example. The last line in the log shows that the build succeeded:

    Creating temporary tarball archive of 2 file(s) totalling 492 bytes before compression.
    Uploading tarball of [.] to [gs://{your project}_cloudbuild/source/1600082502.509759-b949721a922d462c94a75da9be9f1181.tgz]
    Created [https://cloudbuild.googleapis.com/v1/projects/{your project}/builds/ec333452-1301-47e8-90e2-716aeb2f5650].
    Logs are available at [https://console.cloud.google.com/cloud-build/builds/ec333452-1301-47e8-90e2-716aeb2f5650?project=840737568665].
    ------------------------ REMOTE BUILD OUTPUT---------------------------------------
    ...
    ...
    
    Created manifest list gcr.io/{your project}/multiarch-helloworld:latest
    sha256:3ecbbc9f5144f358f81f7c7f1a7e28f069c98423d59c40eaff72bf184af0be02
    2020/09/14 11:34:25 Instance: 35.184.178.49 shut down successfully
    PUSH
    DONE
    -----------------------------------------------------------------------------------
    
    ID                                    CREATE_TIME                DURATION  SOURCE                                                                                      IMAGES  STATUS
    ec333452-1301-47e8-90e2-716aeb2f5650  2020-09-14T11:21:43+00:00  12M43S    gs://{your project}_cloudbuild/source/1600082502.509759-b949721a922d462c94a75da9be9f1181.tgz  -                 SUCCESS
    

You've just built the image using the build config file and pushed the image to Container Registry at gcr.io/{your project}/multiarch-helloworld:latest.

Deploying the image

To deploy the multi-arch Windows image onto a cluster, see Deploying a Windows Server application to learn how to deploy the image.

Advanced gke-windows-builder usage

To see the flags that gke-windows-builder supports, run the following command on a Linux server or in Cloud Shell:

docker run -it gcr.io/gke-release/gke-windows-builder:release-2.0.0-gke.0 --help

To customize the behavior of gke-windows-builder you can add these flags to the args section of the Dockerfile. For example, to speed up your builds you can use a larger machine type for the Windows instances:

  - --machineType
  - 'n1-standard-8'

Instead of building the image for all Windows versions that GKE supports, you can choose specific Windows Server versions to build for by using the --versions flag:

  - --versions
  - '1909,ltsc2019'

If your workspace has many files, your image build will be more reliable if you configure the builder to copy the workspace via Cloud Storage rather than WinRM. Create a bucket in your project, such as gs://{your project}_builder, then set the --workspace-bucket flag:

  - --workspace-bucket
  - '{your project}_builder'

Cleaning up

After you've finished the Building Windows Server multi-arch images tutorial, you can clean up the resources that you created on Google Cloud so they won't take up quota and you won't be billed for them in the future. The following sections describe how to delete or turn off these resources.

Deleting the image

To delete the image:

  1. Open the Container Registry page in the Cloud Console.

    Open the Container Registry page

  2. Go to the Images page.

  3. Click multiarch-helloworld.

  4. Select all images.

  5. Click the Delete button.

  6. Click Delete in the prompt window.

You have now deleted the images that you created as part of this tutorial.

Deleting the project

The easiest way to eliminate billing is to delete the project that you created for the tutorial.

To delete the project:

  1. In the Cloud Console, go to the Manage resources page.

    Go to Manage resources

  2. In the project list, select the project that you want to delete, and then click Delete.
  3. In the dialog, type the project ID, and then click Shut down to delete the project.

What's next