Building the cloud-native future at Google Cloud

From its first open-source commit five years ago to now, Kubernetes has become the industry standard for modern application architecture. It was built on over a decade of Google’s experience as the world’s largest containerized application user. And it’s from this deep and continued investment that Google Cloud provides industry-leading solutions for running workloads at enterprise scale.

One of the most exciting outcomes of this shift toward cloud-native computing is the innovation built on top of Kubernetes. At Google, we love to solve challenging problems, and then share our experiences at scale with the world. This ethos is what brought Kubernetes to life, and it’s also the force behind Knative, Istio, gVisor, Kubeflow, Tekton, and other cloud-native open-source projects that we lead.

We think of it as our job to not only dream about the future, but also to design and implement it. Here’s an overview of open-source projects tied to Kubernetes that we’re working on. We know that speculating about the future can be tricky, but these projects offer a glimpse into how we’re building a cloud-native future. Let’s take a look.

Start with Kubernetes

Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications. It is the industry’s de facto container orchestrator, and is the heart of the cloud-native movement.

We’re proud of our contributions to the Kubernetes project, as we serve the community in many important ways. Google remains the top technical contributor to the project, as well as being actively involved in nearly all special interest groups (SIGs), subprojects, the steering committee, and as code approvers and reviewers. We constantly integrate our real-world experience at scale into the project, just as we have from the beginning.

When we look at the future of Kubernetes, we see the API extension ecosystem maturing and growing even further. We also see a more holistic approach to scalability, so it’s not just about how many nodes or pods are deployed, but how Kubernetes is used across real-world, production environments with widely-varying requirements. Improved reliability is another important facet of this work, as even more mission-critical workloads move to Kubernetes.

Istio

Istio is a service mesh that helps manage, secure and observe traffic between services. The project evolved out of the need for developers adopting microservices to help understand and control the traffic between those services without requiring code changes.

Istio uses the Envoy proxy as a sidecar to collect detailed network traffic statistics and other data from the co-located application, as well as provide logging and tracing. It optionally secures traffic using mTLS (and automatically generates and rotates certificates). Finally, it provides Kubernetes-style APIs to provide advanced networking functionality (for example, the ability to run canary tests, change retry policy at runtime, or add circuit-breaking).

The upcoming version, 1.2, will feature a new operator-based installer and numerous testing and quality improvements. For the rest of 2019, componentization and ease of use will take center stage, as well as architectural improvements that will increase modularity, allow powerful dataplane extensibility, and enhance reliability and performance.

Knative

Knative is a Kubernetes-based platform to build, deploy, and manage modern stateless workloads. Knative components abstract away the complexity and enable developers to focus on what matters to them—solving important business problems.

Just last week, the Knative team released the latest version, v0.6. Besides incremental reliability and stability enhancements, this release also exposes more powerful routing capabilities and improved support for GitOps-like operational use cases. Also, starting with this release, developers can now easily migrate simple apps from Kubernetes Deployments without changes, making service deployment easier for anyone who’s familiar with the Kubernetes resource model.

Since it was announced 10 months ago, a number of commercial offerings already use underlying Knative primitives. Today, the Knative community includes 400+ contributors associated with over 50 different companies, who with the v0.6 release have made 4,000+ pull requests. We are excited about this momentum and look forward to working with the community on further improving the developer experience on Kubernetes.

gVisor

gVisor is an open-source, OCI-compatible sandbox runtime that provides a virtualized container environment. It runs containers with a new user-space kernel, delivering a low-overhead container security solution for high-density applications. gVisor integrates with Docker, containerd and Kubernetes, making it easier to improve the security isolation of your containers while still using familiar tooling. Additionally, gVisor supports a variety of underlying mechanisms for intercepting application calls, allowing it to run in diverse host environments, including cloud-hosted virtual machines.

gVisor was open sourced in May 2018 at KubeCon EU. Since then, the gVisor team has added multi-container support for Kubernetes, released a suite of tests containing more than 1,500 individual tests, released a minikube add-on, integrated it with containerd, and further improved isolation and compatibility. The gVisor team recently began hosting community meetings and is working to grow the users and community around container isolation and gVisor.

Tekton

Tekton is a set of standardized Kubernetes-native primitives for building and running Continuous Delivery workflows. It allows users to express their Continuous Integration, Deployment and Delivery pipelines as Kubernetes CRDs, and run them in any Kubernetes cluster.

We started Tekton last year and donated it to the open Continuous Delivery Foundation earlier this year. Tekton APIs are still in alpha, but we look forward to stabilizing them and adding support for automated deployments, vendor-agnostic pull requests, GitOps workflows, automated compliance-as-code and more!

Forseti Security

Forseti Security is a collection of community-driven, open-source tools to help you expand upon the security of your Google Cloud Platform (GCP) environments. It takes a snapshot of your GCP resources metadata, audits those resources by comparing the configuration with the policies you defined, and notifies you of violations on an ongoing basis.

With Forseti, you can ensure your GKE clusters are provisioned with security and governance guardrails by scanning your GKE resource metadata and making sure the configurations are as expected. Forseti’s Validator Scanner lets you define custom security and governance constraints in Rego to check for violations in your GKE resource metadata.

In addition, you can reuse these constraints for pre-deployment checks with Terraform Validator. A set of canned constraints are available in the Policy Library. The Forseti community will continue contributing new constraints to harden your GKE environment. Get started with Forseti Validator Scanner here.

Kubeflow

Kubeflow is dedicated to making deployments of machine learning (ML) workflows on Kubernetes simple, portable and scalable. Its goal is not to recreate other services, but to provide a straightforward way to deploy best-of-breed open-source systems for ML on a variety of infrastructures. The Kubeflow project is supported by 100+ contributors from 20+ organizations.

Kubeflow is on the road to 1.0, and we're hard at work building a powerful development experience that will allow data scientists to build, train and deploy from notebooks, as well as the enterprise stability and features ML operations teams need to deploy and scale advanced data science workflows. Hear more about this effort in this session from KubeCon NA 2018, and follow us on Twitter @kubeflow.

Skaffold

Skaffold is a command line tool that makes it fast and easy to develop applications on Kubernetes. Skaffold automates the local development loop for you; skaffold dev rebuilds your images and redeploys your app to Kubernetes on every code change. You can also use Skaffold as a building block for CI/CD pipelines with skaffold run. It’s language-agnostic and has an increasing number of configurable, flexible image builders (jib, docker, bazel, kaniko), deployers (kustomize, kubectl, helm) and automated tagging policies, making it a great fit for more and more Kubernetes development workflows.

We use Skaffold under the hood for Cloud Code for IntelliJ and VSCode and also for Jenkins-X. Skaffold is currently in beta, and will soon graduate to 1.0.0.

Follow our progress on our GitHub repo, and share your thoughts with the #skaffold hashtag on Twitter!

Gatekeeper

Gatekeeper is a customizable admission webhook. It allows cluster administrators and security practitioners to develop, share and enforce policies and config validation via parameterized, easily configurable constraint CRDs. Constraints are portable and could also be used to validate commits to the source-of-truth repo in CI/CD pipelines.

With Gatekeeper, you can help developers comply with internal governance and best practices, freeing up your time and theirs. You can do things like require developers to set ownership labels, apply resource limits to their pods, or prohibit them from using the :latest tag. Using Gatekeeper's audit functionality, you can easily find any pre-existing resources that are in violation of current best practices.

Google is proud to be collaborating with Microsoft and Styra (the creators of Open Policy Agent) on this project. Gatekeeper is currently in alpha and we welcome user feedback and contributions.

Krew

Krew is a plugin manager for kubectl that helps users discover and install kubectl plugins to improve their kubectl experiences. Originally developed at Google, Krew is now a part of Kubernetes SIG CLI.  

The future is now

Building cloud-native apps on top of Kubernetes isn’t some abstract, aspirational goal. The tools you need are here today, and they’re only getting better. To stay up to date on what else is happening in the cloud-native community, both from Google and beyond, we urge you to subscribe to the Kubernetes Podcast.