Containers & Kubernetes

Announcing Google Cloud buildpacks—container images made easy

October 9, 2020

Matthew Soldo

Product Manager, Google Workspace Identity

Steren Giannini

Group Product Manager

As a developer building a new application, you want to focus on writing code, not containerizing it. And if you already use containers, you know that creating a good, secure container image can be complicated and time-consuming. Today we’re launching broad support across Google Cloud for buildpacks—an open-source technology that makes it fast and easy for you to create secure, production-ready container images from source code and without a Dockerfile.

At the center of this release are a collection of open-source buildpacks and builders. Based on the CNCF buildpacks v3 specification, these buildpacks produce container images that follow best practices and are suitable for running on all of our container platforms: Cloud Run (fully managed), Anthos, and Google Kubernetes Engine (GKE). These buildpacks are production hardened and tested; they have been used at scale powering most builds for App Engine and Cloud Functions since March.

Using buildpacks

You can use buildpacks in several ways: First, if you haven’t already fully adopted containerization, buildpacks are a way to use the latest container runtime and delivery platforms. They’re also useful for quick projects when you don’t have time to properly vet and customize a Dockerfile that you might find in the wild.

You can try out the Google Cloud Buildpacks with a few clicks by visiting the Buildpack samples project and clicking one of the "Run on Google Cloud" buttons.

To try Buildpacks locally with an application, install Docker and the ’pack’ CLI tool then run:

(Go, Java, Node, Python, and .Net are supported. They might need additional configuration to build properly.)

Or, if you don’t want to install anything, you can run a buildpack based build in Cloud Build, and then easily deploy to Cloud Run:

How do buildpacks work?

Buildpacks are distributed and executed in OCI images called builders. Each builder can have one or more buildpacks. The builder of the Google Cloud buildpacks that we are releasing today is available gcr.io/buildpacks/builder.

Builders have the ability to auto-detect the language of your source code. This is accomplished by a `bin/detect` executable in the buildpack. The detection scripts are invoked in a particular order and stop once an appropriate number of buildpacks have opted in to the build. For example, most Node.js buildpacks look for the presence of a packages.json file. You can also manually specify which buildpack to use, thereby skipping the auto-detection step.

Once the buildpack has been selected, its `bin/build` is executed. This script transforms your source code into an executable artifact, typically performing actions such as installing dependencies or compiling code.

The output of this build step is then added on top of a “run” OCI base image, creating a final container image which can then be run on the platform of your choice.

Google Cloud buildpacks

Google Cloud’s buildpacks are optimized for security, speed, and reusability. They allow you to build both apps and functions into container images. When building a function, they package it using Google Cloud's open-source Functions Framework.

Google Cloud buildpacks use a managed base Ubuntu 18.04 base image that is regularly scanned for security vulnerabilities; any detected vulnerabilities are automatically patched. This ensures that when you build your source code with buildpacks, it will be as secure as possible.

Google Cloud buildpacks can also be customized with additional system packages or to meet your development team’s particular needs.

The buildpacks themselves are all written in Go. Rather than create a single buildpack for each language, you can combine smaller, modular buildpacks together. For example, there is an NPM buildpack which (unsurprisingly) installs node packages. This is of course used for Node.js builds, but it can also be used for other languages and frameworks that use NPM packages (Ruby on Rails, for example).

Broad support in Google Cloud

In addition to the open-source buildpacks, we support buildpacks across a range of our products:

Cloud Build now natively supports buildpacks via the gcloud CLI tool: gcloud alpha builds submit --pack image=gcr.io/[project-id]/my-app. (see documentation)
Cloud Run - Continuous deployment to Cloud Run (via Cloud Build triggers) can be configured to use buildpacks (see documentation).
App Engine - Buildpacks are now the default mechanism for source deployments on most newer App Engine runtimes. Notably, buildpacks enable source-based Java deployments (previously only JAR-based deploys were supported). All newly released runtimes will use buildpacks moving forward.
Cloud Functions - Like App Engine, buildpacks are the default mechanism for building deployed functions.
Cloud Code - Cloud Code IDEs can build your source code with buildpacks and deploy the resulting containers directly to GKE.
Skaffold supports live development with buildpacks. As you edit your source code, buildpacks can continuously re-build your app, allowing you to preview changes in a local instance of your app.
Cloud Shell - The pack CLI tool is now installed in Cloud Shell by default. This allows you to execute buildpacks in Cloud Shell without installing any additional packages.