Connecting from Google Kubernetes Engine

This page describes how to set up a connection from an application running in Google Kubernetes Engine to a Cloud SQL instance.

Introduction

To access a Cloud SQL instance from an application running in Google Kubernetes Engine, you can use either the Cloud SQL Proxy (with public or private IP), or connect directly using a private IP address.

The Cloud SQL Proxy is the recommended way to connect to Cloud SQL, even when using private IP. This is because the proxy provides strong encryption and authentication using IAM, which can help keep your database secure.

Database connections consume resources on the server and the connecting application. Always use good connection management practices to minimize your application's footprint and reduce the likelihood of exceeding Cloud SQL connection limits. For more information, see Managing database connections.

Before you begin

To connect to Cloud SQL you must have:

  • A GKE cluster, with the kubectl command-line tool installed and configured to communicate with the cluster.

    For help getting started with GKE, see the Quickstart.

    For connecting using private IP, the GKE cluster must be VPC-native and in the same VPC network as the Cloud SQL instance.

  • An instance created.

    For help creating a Cloud SQL instance, see Creating Instances.

  • A MySQL user account configured on the instance.

    Your application will use this account to connect to the database. For help with creating a user account, see Creating a user.

About Secrets

In Kubernetes, Secrets are a secure way to pass configuration details to your application. You can create a Secret with details such as your database name, user, and password which can be injected into your application as env vars.

There are many different ways Secrets can be used, depending on the connection type:

  • A database credentials Secret includes the name of the database user you are connecting as, and the user's database password.
  • If connecting with the proxy, a Secret can be used to hold your service account's credential file.
  • If connecting with private IP, a Secret can be used to specify the private IP address of your Cloud SQL instance.

For complete examples of how to use Secrets, see the GitHub repositories referenced later on this page.

Creating a Secret object

  1. You create the Secret objects by using the kubectl create secret command.

    To create a database credentials Secret:

    kubectl create secret generic <YOUR-DB-SECRET> \
      --from-literal=username=<YOUR-DATABASE-USER> \
      --from-literal=password=<YOUR-DATABASE-PASSWORD> \
      --from-literal=database=<YOUR-DATABASE-NAME>
    
  2. Once created, you can view the objects in the Configuration section of the Google Kubernetes Engine page in the Cloud Console.

Connecting using the Cloud SQL Proxy

When you connect using the Cloud SQL Proxy, the Cloud SQL Proxy is added to your pod using the sidecar container pattern. The proxy container is in the same pod as your application, which enables the application to connect to the proxy using localhost, increasing security and performance. Learn more.

For more information about the Cloud SQL Proxy, see About the Cloud SQL Proxy. For more information about working with pods, see Pod Overview in the Kubernetes documentation.

For connecting using the Cloud SQL Proxy you need the following:

  1. The instance connection name of your Cloud SQL instance.

    The instance connection name is available in the Cloud SQL Instance details page of the Cloud Console or from the gcloud sql instances describe command.

  2. The location of the key file associated with a service account with the proper privileges for your Cloud SQL instance.

    See Creating a service account for more information.

  3. The Cloud SQL Admin API is enabled.

    Enable the API

Providing the service account to the proxy

The first step to running the Cloud SQL Proxy in Google Kubernetes Engine is creating a service account to represent your application. It is recommended that you create a service account unique to each application, instead of using the same service account everywhere. This model is more secure since it allows your to limit permissions on a per-application basis.

The service account for your application needs to meet the following criteria:

  • Belong to a project with the Cloud SQL Admin API enabled
  • Has been granted the Cloud SQL Client IAM role (or equivalent) for the project containing the instance you want to connect to
  • If connecting using private IP, you must use a VPC-native GKE cluster, in the same VPC as your Cloud SQL instance

You need to configure GKE to provide the service account to the Cloud SQL Proxy. There are two recommended ways to do this: workload identity or a service account key file.

Workload Identity

If you are using Google Kubernetes Engine, the preferred method is to use GKE's Workload Identity feature. This method allows you to bind a Kubernetes Service Account (KSA) to a Google Service Account (GSA). The GSA will then be accessible to applications using the matching KSA.

A Google Service Account (GSA) is an IAM identity that represents your application in Google Cloud. In a similar fashion, a Kubernetes Service Account (KSA) is a an identity that represents your application in a Google Kubernetes Engine cluster.

Workload Identity binds a KSA to a GSA, causing any deployments with that KSA to authenticate as the GSA in their interactions with Google Cloud.

  1. Enable Workload Identity for your cluster
  2. Typically, each application has its own identity, represented by a KSA and GSA pair. Create a KSA for your application by running kubectl apply -f service-account.yaml:

    apiVersion: v1
    kind: ServiceAccount
    metadata:
      name: <YOUR-KSA-NAME> # TODO(developer): replace these values
  3. Enable the IAM binding between your YOUR-GSA-NAME and YOUR-KSA-NAME:

    gcloud iam service-accounts add-iam-policy-binding \
      --role roles/iam.workloadIdentityUser \
      --member "serviceAccount:<YOUR-GCP-PROJECT>.svc.id.goog[<YOUR-K8S-NAMESPACE>/<YOUR-KSA-NAME>]" \
      <YOUR-GSA-NAME>@<YOUR-GCP-PROJECT>.iam.gserviceaccount.com
    
  4. Add an annotation to YOUR-KSA-NAME to complete the binding:

    kubectl annotate serviceaccount \
       <YOUR-KSA-NAME> \
       iam.gke.io/gcp-service-account=<YOUR-GSA-NAME>@<YOUR-GCP-PROJECT>.iam.gserviceaccount.com
    
  5. Finally, make sure to specify the service account for the k8s object.

    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: <YOUR-DEPLOYMENT-NAME>
    spec:
      selector:
        matchLabels:
          app: <YOUR-APPLICATION-NAME>
      template:
        metadata:
          labels:
            app: <YOUR-APPLICATION-NAME>
        spec:
          serviceAccountName: <YOUR-KSA-NAME>

Service account key file

Alternatively, if you can't use Workload Identity, the recommended pattern is to mount a service account key file into the Cloud SQL proxy pod and use the -credential_file flag.

  1. Create a credential file for your service account key:

    gcloud iam service-accounts keys create ~/key.json \
      --iam-account <YOUR-SA-NAME>@project-id.iam.gserviceaccount.com
    
  2. Turn your service account key into a k8s Secret:

    kubectl create secret generic <YOUR-SA-SECRET> \
    --from-file=service_account.json=~/key.json
    
  3. Mount the secret as a volume under the spec: for your k8s object:

    volumes:
    - name: <YOUR-SA-SECRET-VOLUME>
      secret:
        secretName: <YOUR-SA-SECRET>
    
  4. Follow the instructions in the next section to access the volume from the proxy's pod.

Running the proxy as a sidecar

We recommend running the proxy in a sidecar pattern (as an additional container sharing a pod with your application). We recommend this over running as a separate service for several reasons:

  • Prevents your SQL traffic from being exposed locally; the proxy provides encryption on outgoing connections, but you need to limit exposure for incoming connections
  • Prevents a single point of failure; each application's access to your database is independent from the others, making it more resilient.
  • Limits access to the proxy, allowing you to use IAM permissions per application rather than exposing the database to the entire cluster
  • Allows you to scope resource requests more accurately; because the proxy consumes resources linearly to usage, this pattern allows you to more accurately scope and request resources to match your applications as it scales
  1. Add the Cloud SQL proxy to the pod configuration under containers::

    - name: cloud-sql-proxy
      # It is recommended to use the latest version of the Cloud SQL proxy
      # Make sure to update on a regular schedule!
      image: gcr.io/cloudsql-docker/gce-proxy:1.17
      command:
        - "/cloud_sql_proxy"
    
        # If connecting from a VPC-native GKE cluster, you can use the
        # following flag to have the proxy connect over private IP
        # - "-ip_address_types=PRIVATE"
    
        # Replace DB_PORT with the port the proxy should listen on
        # Defaults: MySQL: 3306, Postgres: 5432, SQLServer: 1433
        - "-instances=<INSTANCE_CONNECTION_NAME>=tcp:<DB_PORT>"
      securityContext:
        # The default Cloud SQL proxy image runs as the
        # "nonroot" user and group (uid: 65532) by default.
        runAsNonRoot: true

    If you are using a service account key, specify your secret volume and add the -credential_file flag to the command:

      # This flag specifies where the service account key can be found
      - "-credential_file=/secrets/service_account.json"
    securityContext:
      # The default Cloud SQL proxy image runs as the
      # "nonroot" user and group (uid: 65532) by default.
      runAsNonRoot: true
    volumeMounts:
    - name: <YOUR-SA-SECRET-VOLUME>
      mountPath: /secrets/
      readOnly: true
  2. Finally, configure your application to connect using 127.0.0.1 on whichever DB_PORT you specified in the command section.

Complete example configuration files:

Proxy with Workload Identity

apiVersion: apps/v1
kind: Deployment
metadata:
  name: <YOUR-DEPLOYMENT-NAME>
spec:
  selector:
    matchLabels:
      app: <YOUR-APPLICATION-NAME>
  template:
    metadata:
      labels:
        app: <YOUR-APPLICATION-NAME>
    spec:
      serviceAccountName: <YOUR-KSA-NAME>
      containers:
      - name: <YOUR-APPLICATION-NAME>
        # ... other container configuration
        env:
        - name: DB_USER
          valueFrom:
            secretKeyRef:
              name: <YOUR-DB-SECRET>
              key: username
        - name: DB_PASS
          valueFrom:
            secretKeyRef:
              name: <YOUR-DB-SECRET>
              key: password
        - name: DB_NAME
          valueFrom:
            secretKeyRef:
              name: <YOUR-DB-SECRET>
              key: database
      - name: cloud-sql-proxy
        # It is recommended to use the latest version of the Cloud SQL proxy
        # Make sure to update on a regular schedule!
        image: gcr.io/cloudsql-docker/gce-proxy:1.17
        command:
          - "/cloud_sql_proxy"

          # If connecting from a VPC-native GKE cluster, you can use the
          # following flag to have the proxy connect over private IP
          # - "-ip_address_types=PRIVATE"

          # Replace DB_PORT with the port the proxy should listen on
          # Defaults: MySQL: 3306, Postgres: 5432, SQLServer: 1433
          - "-instances=<INSTANCE_CONNECTION_NAME>=tcp:<DB_PORT>"
        securityContext:
          # The default Cloud SQL proxy image runs as the
          # "nonroot" user and group (uid: 65532) by default.
          runAsNonRoot: true

Proxy with Service Account Key

apiVersion: apps/v1
kind: Deployment
metadata:
  name: <YOUR-DEPLOYMENT-NAME>
spec:
  selector:
    matchLabels:
      app: <YOUR-APPLICATION-NAME>
  template:
    metadata:
      labels:
        app: <YOUR-APPLICATION-NAME>
    spec:
      containers:
      - name: <YOUR-APPLICATION-NAME>
        # ... other container configuration
        env:
        - name: DB_USER
          valueFrom:
            secretKeyRef:
              name: <YOUR-DB-SECRET>
              key: username
        - name: DB_PASS
          valueFrom:
            secretKeyRef:
              name: <YOUR-DB-SECRET>
              key: password
        - name: DB_NAME
          valueFrom:
            secretKeyRef:
              name: <YOUR-DB-SECRET>
              key: database
      - name: cloud-sql-proxy
        # It is recommended to use the latest version of the Cloud SQL proxy
        # Make sure to update on a regular schedule!
        image: gcr.io/cloudsql-docker/gce-proxy:1.17
        command:
          - "/cloud_sql_proxy"

          # If connecting from a VPC-native GKE cluster, you can use the
          # following flag to have the proxy connect over private IP
          # - "-ip_address_types=PRIVATE"

          # Replace DB_PORT with the port the proxy should listen on
          # Defaults: MySQL: 3306, Postgres: 5432, SQLServer: 1433
          - "-instances=<INSTANCE_CONNECTION_NAME>=tcp:<DB_PORT>"

          # This flag specifies where the service account key can be found
          - "-credential_file=/secrets/service_account.json"
        securityContext:
          # The default Cloud SQL proxy image runs as the
          # "nonroot" user and group (uid: 65532) by default.
          runAsNonRoot: true
        volumeMounts:
        - name: <YOUR-SA-SECRET-VOLUME>
          mountPath: /secrets/
          readOnly: true
      volumes:
      - name: <YOUR-SA-SECRET-VOLUME>
        secret:
          secretName: <YOUR-SA-SECRET>

Connecting without the Cloud SQL proxy

While not as secure, it is possible to connect from a VPC-native GKE cluster to a Cloud SQL instance on the same VPC using private IP without the proxy.

  1. Create a secret with your instance's private IP address:

    kubectl create secret generic <YOUR-PRIVATE-IP-SECRET> \
        --from-literal=db_host=<YOUR-PRIVATE-IP-ADDRESS>
    
  2. Next make sure you add the secret to your application's container:

    - name: DB_HOST
      valueFrom:
        secretKeyRef:
          name: <YOUR-PRIVATE-IP-SECRET>
          key: db_host
  3. Finally, configure your application to connect using the IP address from the DB_HOST env var. You will need to use the correct port for MySQL: 3306

Complete example configuration file:

No Proxy - Private IP

apiVersion: apps/v1
kind: Deployment
metadata:
  name: <YOUR-DEPLOYMENT-NAME>
spec:
  selector:
    matchLabels:
      app: <YOUR-APPLICATION-NAME>
  template:
    metadata:
      labels:
        app: <YOUR-APPLICATION-NAME>
    spec:
      containers:
      - name: <YOUR-APPLICATION-NAME>
        # ... other container configuration
        env:
        - name: DB_USER
          valueFrom:
            secretKeyRef:
              name: <YOUR-DB-SECRET>
              key: username
        - name: DB_PASS
          valueFrom:
            secretKeyRef:
              name: <YOUR-DB-SECRET>
              key: password
        - name: DB_NAME
          valueFrom:
            secretKeyRef:
              name: <YOUR-DB-SECRET>
              key: database
        - name: DB_HOST
          valueFrom:
            secretKeyRef:
              name: <YOUR-PRIVATE-IP-SECRET>
              key: db_host

Need help? For help troubleshooting the proxy, see Troubleshooting Cloud SQL Proxy connections. Or, see our Cloud SQL Support page.

What's next