Exploring container security: Digging into Grafeas container image metadata

Editor’s note: This is the third in a series of blog posts on container security at Google.

The great thing about containers is how easy they are to create, modify and share. But that also raises the question of whether or not they're safe to deploy to production. One way to answer that is to track metadata about your container, for example, who worked on it, where it's stored, and whether it has any known vulnerabilities.

Last October, Google and several partners announced Grafeas, an open-source project that provides a structured metadata API for container images. Grafeas simplifies how you handle metadata, including security concerns like package vulnerability scan results, and keeps track of lots of different kinds of information:

• "Build" metadata can be used to certify that a container image was built according to your build policies using a trusted builder with checked-in source code. 
• "Package Vulnerability" metadata contains information from a vulnerability scanning service and lets you create policies on deploying vulnerable containers. 
• "Image Basis" metadata contains information about the base image from which your container was derived as well as additional layers used in building the container image. 
• "Package Manager" metadata indicates which packages were installed in the container image. 
• "Deployment History" metadata allows you to track which resources were deployed and when.

In fact, you can use Grafeas to enforce all kinds of security policies. Check out the tutorial by Kelsey Hightower on how to use Grafeas to allow only container images signed by a specific private key. But it doesn’t stop there. For example, you can write policies to block container images with vulnerabilities from being deployed, ensure that all your deployed images are built from a base image sanctioned by your security team, or require images to go through your build pipeline. The possibilities are endless!

How Shopify uses Grafeas to manage metadata for its 500k container images

Grafeas lets us answer fundamental questions like:

When was the container created? What code runs inside it? Who created it? Where is the running container located? Is the container vulnerable to a known exploit and do we need to pull it out of production to replace it?

Shopify’s Continuous Integration/Deployment (CI/CD) pipeline supplies information about who built the code, when it was built, what the commit hash is, and whether or not tests passed. We store this data in the “build” metadata in Grafeas, leaving a digital paper trail that can be audited to track changes.

After build, we test the container for known vulnerabilities. Since that list of vulnerabilities can change at any time, we check our images at regular intervals. The information in Grafeas’ “package vulnerability” metadata field is updated at each check and allows us to make informed decisions about when to pull an image out of production and deploy a new secure one.

Next up, we’re working on how to track information about deploys in the deployment history metadata field, so that we know when, where and by whom a container was deployed, as well as its current status. We will use that information to create a “bill of materials” of what’s running in our environment.

The metadata in Grafeas gives us an answer to all of our questions around deploys. We also use it to restrict the deployment of containers. During the creation of a workload, the admission controller makes its decision by querying Grafeas for the container’s metadata — if it cannot find anything that violates predefined policies, for example the existence of known vulnerabilities, it allows the workload to be created.

Using Grafeas in production gives us a 360-degree view of our deploys so we can protect our infrastructure and maintain a history of what’s going on in our cloud.

Get involved in Grafeas 

If you’d like to contribute, Grafeas is always looking for more contributors as we grow this community. Join us!