Cloud Run and Kubernetes both use standard container images as deployment artifacts, they both use a declarative API model with resources that can be represented in YAML files with the same standard structure.
Introduction
The Cloud Run Admin API v1 is designed to maximize portability with Kubernetes, for example, the Cloud Run Admin API resources share the same structure conventions and attribute names as Kubernetes resources. See the Cloud Run service YAML reference.
The Cloud Run Admin API v1 implements the Knative Serving API specification, but you do not need to be using Knative in your existing Kubernetes cluster in order to migrate some of your Kubernetes workloads to Cloud Run.
The primary resource of Cloud Run is the service. You can think of a Cloud Run service as a high level abstraction that looks like a Kubernetes deployment with a built-in pod autoscaler and unique endpoint. A "Pod" in Kubernetes corresponds to an "instance" in Cloud Run. We recommend transforming your Kubernetes deployments into Cloud Run services, one service at a time. You will also be able to merge some configuration of your Kubernetes Horizontal Pod Autoscaler and Kubernetes services into the Cloud Run service.
Cloud Run does not have the concept of namespace, instead the Google Cloud project is used as an isolation boundary between resources. When migrating to Cloud Run from Kubernetes, we recommend creating one Google Cloud project for each namespace. In the YAML of a Cloud Run service, the namespace value is the Google Cloud project number.
Cloud Run has a built-in zonal redundancy, this means that you do not need to provision replica to ensure your service is resilient to a zonal outage in the selected Google Cloud region.
Quickstart
This quickstart is an example of a simple migration example.
Simple resource comparison
Compare the following simple Kubernetes deployment
named my-app
with the equivalent Cloud Run service.
Note how the YAML files are almost identical.
The parts in blue
are different and
need to be changed.
The parts in red
should be deleted
because Cloud Run has a built-in autoscaler.
Kubernetes deployment | Cloud Run service |
---|---|
apiVersion: apps/v1 kind: Deployment metadata: name: my-app namespace: default labels: app: my-app spec: template: metadata: labels: app: my-app spec: containers: - image: gcr.io/cloudrun/hello env: - name: HELLO value: world replicas: 3 selector: matchLabels: app: my-app |
apiVersion: serving.knative.dev/v1 kind: Service metadata: name: my-app namespace: 'PROJECT_NUMBER' labels: app: my-app spec: template: metadata: labels: app: my-app spec: containers: - image: gcr.io/cloudrun/hello env: - name: HELLO value: world |
Migrate a simple Kubernetes deployment to Cloud Run
Download the YAML file of your deployment in the current directory with:
kubectl get deployment my-app -o yaml > my-app.yaml
Modify the YAML to match a Cloud Run service. Update the
my-app.yaml
file:- For the "
kind
" attribute: replace the value "Deployment
" with "Service
" - For the "
apiVersion
" attribute: replace the value "apps/v1
" with "serving.knative.dev/v1
" - Delete the
metadata.namespace
attribute or change its value to match your Google Cloud project number. - Delete
spec.replicas
andspec.selector
- For the "
Deploy the
my-app.yaml
file to Cloud Run using the following command, replacing REGION with the desired Google Cloud region, for exampleus-central1
:gcloud run services replace my-app.yaml --region REGION
Invoking a Cloud Run service is protected by an IAM permission. If you want to expose the new Cloud Run service publicly to the internet and allow unauthenticated invocations, run the following command:
gcloud run services add-iam-policy-binding my-app --member="allUsers" --role="roles/run.invoker" --region REGION
Features not supported by Cloud Run
Only workloads that are a fit for Cloud Run can be migrated.
Notably, the following Kubernetes features are not supported by Cloud Run:
- Fixed numbers of
replicas
(a workaround is to use the same number of minimum and maximum instances - Config Maps (workaround available)
- Custom horizontal Pod autoscaler strategies
- Service Discovery
When migrating a YAML file from a Kubernetes deployment to a Cloud Run
service, the gcloud run services replace
command will return a clear error
message for any attribute that is not supported by Cloud Run.
Delete or update these attributes, then repeat the command until it succeeds.
You can refer to the YAML Reference for an exhaustive list of attributes support by Cloud Run.
Migrate Kubernetes resources
Migrate Kubernetes secrets
Like Kubernetes, Cloud Run supports mounting secrets as environment variables or volumes, but secrets should be stored in Secret Manager.
There are several important differences between Secret Manager secrets and Kubernetes secrets:
- Allowed characters in names:
- Kubernetes secrets:
[a-z0-9-.]{1,253}
- Secret Manager secrets:
[a-zA-Z0-9_-]{1,255}
- Kubernetes secrets:
- Versioning: Secrets from Secret Manager are versioned, while Kubernetes secrets aren't.
- Payload: Secrets from Secret Manager contain a single
[]byte
, while Kubernetes secrets contain amap<string, string>
.
Follow the Secret Manager documentation to create a secret and add a new secret version for every secret key your Kubernetes app depends on.
Migrate Kubernetes ConfigMaps
Cloud Run does not have an equivalent of Kubernetes ConfigMaps, but because ConfigMaps can be seen as unencrypted Secrets, you can transform your ConfigMaps into secrets in Secret Manager instead. See instructions under Migrate Kubernetes secrets.
Migrate a Kubernetes deployment
The Kubernetes deployment is the resource that maps the closest to the Cloud Run service. We recommend starting from the YAML of your Kubernetes deployment and edit it to transform it into a Cloud Run service.
The main required changes are:
- The
namespace
must be replaced with the Google Cloud project number. - Labels (
metadata.labels
andspec.template.metadata.labels
) must be valid Google Cloud labels. - Containers must be stored in a supported container registry.
- CPU and memory limits might need to be adjusted.
- When referencing a secret, the "
key
" attribute is used to capture the version in Secret Manager, with "latest
" referencing the latest version of the secret. serviceAccountName
should reference a Service Account in the current Google Cloud project- References to ConfigMaps (
configMapKeyRef
) should be replaced with references to secrets (secretKeyRef
)
If your Kubernetes deployment is accessing other resources in your Kubernetes cluster or resources on a VPC, you must connect the Cloud Run service to the appropriate VPC
Migrate a Kubernetes Service
Cloud Run services automatically expose a unique endpoint that routes
traffic to the container with a containerPort
.
After you have migrated your Kubernetes deployment to a Cloud Run
service, you don't need to migrate the Kubernetes services
that was routing traffic to this deployment.
Migrate a Kubernetes HorizontalPodAutoscaler
Cloud Run services have a built-in horizontal autoscaler: Cloud Run automatically scales pods (called "instances") using a combination of factors within the boundaries of its defined minimum and maximum number of instances.
Migrate the minReplicas
and maxReplicas
attributes of the HorizontalPodAutoscaler
to the autoscaling.knative.dev/minScale
and autoscaling.knative.dev/maxScale
annotations of the Cloud Run service.
Refer to the documentation on configuring
minimum instances and maximum instances.
Migrate a Kubernetes Job
Because a Kubernetes job is similar to a Cloud Run job execution. You can migrate to a Cloud Run job and execute the job.
The following samples show the structural difference between a Kubernetes job and a Cloud Run job:
Kubernetes job | Cloud Run job |
---|---|
apiVersion: batch/v1
kind: Job
metadata:
name: my-job
spec:
template:
spec:
containers:
- image: us-docker.pkg.dev/cloudrun/container/job
|
apiVersion: run.googleapis.com/v1 kind: Job metadata: name: my-job spec: template: spec: template: spec: containers: - image: us-docker.pkg.dev/cloudrun/container/job |
Migration strategy
After you create the equivalent resources, exposing external endpoints behind a global external Application Load Balancer enables you to gradually migrate traffic between Cloud Run and Google Kubernetes Engine (GKE).