Unify Kubernetes and GCP resources for simpler and faster deployments

Adopting containers and Kubernetes means adopting new ways of doing things, not least of which is how you configure and maintain your resources. As a declarative system, Kubernetes allows you to express your intent for a given resource, and then creates and updates those resources using continuous reconciliation. Compared with imperative configuration approaches, Kubernetes-style declarative config helps ensure that your organization follows GitOps best practices like storing configuration in a version control system, and defining it in a YAML file.   

However, applications that run on Kubernetes often use resources that live outside of Kubernetes, for example, Cloud SQL or Cloud Storage, and those resources typically don’t use the same approach to configuration. This can cause friction between teams, and force developers into frequent “context switching”. Further, configuring and operating those applications is a multi-step process: configuring the external resources, then the Kubernetes resources, and finally making the former available to the latter. 

To help, today, we're announcing the general availability of Config Connector, which lets you manage Google Cloud Platform (GCP) resources as Kubernetes resources, giving you a single place to configure your entire application.

Config Connector is a Kubernetes operator that makes all GCP resources behave as if they were Kubernetes resources, so you don't have to learn and use multiple conventions and tools to manage your infrastructure. For cloud-native developers, Config Connector simplifies operations and resource management by providing a uniform and consistent way to manage all of cloud infrastructure through Kubernetes.

Automating infrastructure consistency

With its declarative approach, Kubernetes is continually reconciling the resources it manages. Resources managed by Kuberentes are continuously monitored, and “self-heal” to continuously meet the user’s desired state. However, monitoring and reconciliation of non-Kubernetes resources (a SQL server instance for example), happens as part of a separate workflow. In the most extreme cases, changes to your desired configuration, for example, changes to the number of your Cloud Spanner nodes, are not propagated to your monitoring and alerting infrastructure, causing false alarms and creating additional work for your teams. 

By bringing these resources under the purview of Kuberentes with Config Connector, you get resource reconciliation across your infrastructure, automating the work of achieving eventual consistency in your infrastructure. Instead of spinning up that SQL server instance separately and monitoring it for changes as a second workflow, you ask Config Connector to create a SQL server instance and an SQL database on that instance. Config Connector creates these resources, and now that they’re part of your declarative approach, the SQL server instance is effectively self-healing, just like the rest of your Kubernetes deployment. 

Using Kubernetes’ resource model relieves you from having to explicitly order resources in your deployment scripts. Just like for pods, deployments, or other native Kubernetes resources, you no longer have to explicitly wait for the SQL instance to be completed before starting to provision an SQL database on that instance, as illustrated in the YAML manifests below.

Additionally, by defining GCP resources as Kubernetes objects, you now get to leverage familiar Kubernetes features with these resources, such as Kubernetes Labels and Selectors. For example, here  we used cost-center as a label on the resources. You can now filter by this label using kubectl get. Furthermore, you can apply your organization’s governance policy using admission controllers, such as Anthos Policy Controller. For example, you can enforce that the cost-center label should exist on all resources in the cluster and only have an allowed range of values:

Faster development with simplified operations

For Etsy, Kubernetes was instrumental in helping them to move to the cloud, but the complexity of their applications meant they were managing resources in multiple places, slowing down their deployments.

“At Etsy, we run complex Kubernetes applications that combine custom code and cloud resources across many environments. Config Connector will allow Etsy to move from having two distinct, disconnected CI/CD pipelines to a single pipeline for both application code and the infrastructure it requires. Config Connector will simplify our delivery and enable end-to-end testing of cloud infrastructure changes, which we expect will result in faster deployment and lower friction usage of cloud infrastructure” - Gregg Donovan, Senior Staff Software Engineer, Etsy. 

Getting started with Config Connector

Today, Config Connector can be used to manage more than 60+ GCP services, including Bigtable, BigQuery, IAM Policies, Service Account and Service Account Keys, Pub/Sub, Redis, Spanner, Cloud SQL, Cloud Storage, Compute Engine, Networking and Cloud Load Balancer. 

Config Connector can be installed standalone on any Kubernetes cluster, and is also integrated into Anthos Config Management, for managing hybrid and multi-cloud environments. Get started with Config Connector today to simplify configuration management across GKE and GCP.