This page explains how to deploy a stateful application using Google Kubernetes Engine (GKE).
Overview
Stateful applications save data to persistent disk storage for use by the server, by clients, and by other applications. An example of a stateful application is a database or key-value store to which data is saved and retrieved by other applications.
Persistent storage can be dynamically provisioned, so that the underlying volumes are created on demand. In Kubernetes, you configure dynamic provisioning by creating a StorageClass. In GKE, a default StorageClass allows you to dynamically provision Compute Engine persistent disks.
Kubernetes uses the StatefulSet controller to deploy stateful applications as StatefulSet objects. Pods in StatefulSets are not interchangeable: each Pod has a unique identifier that is maintained no matter where it is scheduled.
Stateful applications are different from stateless applications, in which client data is not saved to the server between sessions.
You can learn more about persistent storage in multi-zonal and regional clusters.
Before you begin
Before you start, make sure you have performed the following tasks:
- Ensure that you have enabled the Google Kubernetes Engine API. Enable Google Kubernetes Engine API
- Ensure that you have installed the Cloud SDK.
Set up default gcloud
settings using one of the following methods:
- Using
gcloud init
, if you want to be walked through setting defaults. - Using
gcloud config
, to individually set your project ID, zone, and region.
Using gcloud init
If you receive the error One of [--zone, --region] must be supplied: Please specify
location
, complete this section.
-
Run
gcloud init
and follow the directions:gcloud init
If you are using SSH on a remote server, use the
--console-only
flag to prevent the command from launching a browser:gcloud init --console-only
-
Follow the instructions to authorize
gcloud
to use your Google Cloud account. - Create a new configuration or select an existing one.
- Choose a Google Cloud project.
- Choose a default Compute Engine zone for zonal clusters or a region for regional or Autopilot clusters.
Using gcloud config
- Set your default project ID:
gcloud config set project PROJECT_ID
- If you are working with zonal clusters, set your default compute zone:
gcloud config set compute/zone COMPUTE_ZONE
- If you are working with Autopilot or regional clusters, set your default compute region:
gcloud config set compute/region COMPUTE_REGION
- Update
gcloud
to the latest version:gcloud components update
- Ensure your containerized application is stored in an image registry, such as Container Registry.
You can follow the quickstart, to enable the GKE API, create a cluster, and learn more about GKE.
Requesting persistent storage in a StatefulSet
Applications can request persistent storage with a PersistentVolumeClaim.
Normally, PersistentVolumeClaim objects have to be created by the user in
addition to the Pod. However, StatefulSets include a volumeClaimTemplates
array, which automatically generates the PersistentVolumeClaim objects. Each
StatefulSet replica gets its own PersistentVolumeClaim object.
You can also use a preexisting disk in a StatefulSet.
Creating a StatefulSet
To create a StatefulSet, use the kubectl apply
command.
The kubectl apply
command uses manifest files to create, update, and delete
resources in your cluster. This is a declarative
method
of object configuration. This method retains writes made to live objects without
merging the changes back into the object configuration files.
Linux
The following manifest file is a simple example of a StatefulSet governed by a Service that has been created separately:
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: statefulset-name
spec:
selector:
matchLabels:
app: app-name
serviceName: "service-name"
replicas: 3
updateStrategy:
type: RollingUpdate
template:
metadata:
labels:
app: app-name
spec:
containers:
- name: container-name
image: ...
ports:
- containerPort: 80
name: port-name
volumeMounts:
- name: pvc-name
mountPath: ...
volumeClaimTemplates:
- metadata:
name: pvc-name
annotations:
...
spec:
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 1Gi
Where:
- statefulset-name is the name of the StatefulSet.
- service-name is the name of the Service.
- app-name is the name of the application run in the Pods.
- container-name is the name of the containers in the Pods.
- port-name is the name of the port opened by the StatefulSet.
- pvc-name is the name of the PersistentVolumeClaim.
In this file, the kind
field specifies that a StatefulSet object should be
created with the specifications defined in the file. This example StatefulSet
produces three replicated Pods, and opens port 80 for exposing the
StatefulSet to the internet.
Windows
When using clusters with Windows Server node pools, you must create a StorageClass, as the default StorageClass uses ext4 as the file system type, which only works for Linux containers. If you are using a Compute Engine persistent disk, you must use NTFS as the file storage type as shown in the following example:
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: storageclass-name
parameters:
type: pd-standard
fstype: NTFS
provisioner: kubernetes.io/gce-pd
reclaimPolicy: Delete
volumeBindingMode: WaitForFirstConsumer
The following StatefulSet manifest uses the StorageClass you defined above. It creates four PersistentVolume and PersistentVolumeClaim pairs to represent four Compute Engine persistent disks. Each Pod in the StatefulSet consumes one persistent disk.
To ensure your Pods are correctly scheduled onto Windows Server nodes, you must add a node selector to your Pod specification.
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: statefulset-name
spec:
replicas: 4
selector:
matchLabels:
app: app-name
template:
metadata:
labels:
app: app-name
name: container-name
spec:
nodeSelector:
kubernetes.io/os: windows
containers:
- name: container-name
image: ...
ports:
- containerPort: 80
name: port-name
volumeMounts:
- name: pvc-name
mountPath: C:\mnt\state
volumeClaimTemplates:
- metadata:
name: pvc-name
spec:
storageClassName: storageclass-name
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 1Gi
Where:
- app-name is the name of the application run in the Pods.
- statefulset-name is the name of the StatefulSet.
- container-name is the name of the containers in the Pods.
- port-name is the name of the port opened by the StatefulSet.
- pvc-name is the name of the PersistentVolumeClaim.
- storageclass-name is the name of the StorageClass.
To create a StatefulSet, run the following command:
kubectl apply -f statefulset-file
where statefulset-file is the manifest file.
You can also use kubectl apply -f directory/
to create
all objects (except existing ones) defined in configuration files stored in a
directory.
Inspecting a StatefulSet
kubectl
To inspect the StatefulSet, run the following command:
kubectl get statefulset statefulset-name -o yaml
This command displays the StatefulSet's live configuration in YAML format.
To list the Pods created by the StatefulSet, run the following command:
kubectl get pods -l app=app-name
In this command, the -l
flag instructs kubectl
to get all Pods labeled
for the app-name.
The output is similar to the following:
NAME READY STATUS RESTARTS AGE pod-name 1/1 Running 0 1m pod-name 1/1 Running 0 1m
To get detailed information about the StatefulSet, run the following command:
kubectl describe statefulset statefulset-name
where statefulset-name is the name of the StatefulSet.
To get information about a specific Pod:
kubectl describe pod pod-name
To list the PersistentVolumeClaims that were created:
kubectl get pvc
The output is similar to the following:
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE statefulset-name-pvc-name-0 Bound pvc-bdff4e1e-183e-11e8-bf6d-42010a800002 1G RWO standard 9s statefulset-name-pvc-name-1 Bound pvc-bdff4e1e-183e-11e8-bf6d-42010a800003 1G RWO standard 9s statefulset-name-pvc-name-2 Bound pvc-bdff4e1e-183e-11e8-bf6d-42010a800004 1G RWO standard 9s
To get information about a specific PersistentVolumeClaim:
kubectl describe pvc statefulset-name-pvc-name-0
To get information about a specific PersistentVolume:
kubectl describe pv pv-name
Console
To inspect a StatefulSet, perform the following steps:
Visit the Google Kubernetes Engine Workloads menu in Cloud Console.
In the workloads list, click the name of the StatefulSet you want to inspect.
On the Stateful Set details page, do any of the following:
- Click the Revision History tab to see the StatefulSet's revision history.
- Click the Events tab to see all events related to the StatefulSet.
- Click the Logs tab to see container logs for the StatefulSet.
- Click the YAML tab to see, copy, or download the configuration YAML for the StatefulSet.
Updating a StatefulSet
There are multiple ways of updating StatefulSets. The common, declarative method
is kubectl apply
. To update the StatefulSet directly from your shell or in a
preferred editor, you can use kubectl edit
. You can also use the YAML editor
from the GKE Workloads menu in the Cloud Console.
You can roll out updates to a StatefulSet's Pods specification such as its image, resource usage/requests, or configuration.
kubectl apply
You can update the StatefulSet by applying a new or updated manifest file. This is useful for making various changes to your StatefulSet, such as when scaling or for specifying a new version of your application.
To update a StatefulSet, run the following command:
kubectl apply -f statefulset-file
where statefulset-file is the updated manifest file.
The kubectl apply
command applies a manifest file to a resource. If the
specified resource does not exist, it is created by the command.
For more information about kubectl apply
, see the
kubectl
reference documentation.
Console
To edit the live configuration of a StatefulSet, perform the following steps:
Visit the Google Kubernetes Engine Workloads menu in Cloud Console.
In the workloads list, click the name of the StatefulSet you want to modify.
Click edit Edit.
Change the configuration YAML as desired.
Click Save.
Inspecting update rollout
kubectl
To inspect the StatefulSet's rollout, run the following command:
kubectl rollout status statefulset statefulset-name
To see the StatefulSet's rollout history:
kubectl rollout history statefulset statefulset-name
To undo a rollout:
kubectl rollout undo statefulset statefulset-name
Console
To see the revision history of a StatefulSet, perform the following steps:
Visit the Google Kubernetes Engine Workloads menu in Cloud Console.
In the workloads list, click the name of the StatefulSet you want to inspect.
Click the Revision History tab.
Select the desired revision.
Update strategies
StatefulSet’s updateStrategy
field allows you to configure and disable
automated rolling updates for containers, labels, resource request/limits, and
annotations for the Pods in a StatefulSet.
You can learn more about Update Strategies for StatefulSets in the Kubernetes documentation.
Scaling a StatefulSet
kubectl
kubectl scale
can be used at any time to scale your StatefulSet.
To manually scale a StatefulSet, run the following command:
kubectl scale statefulset statefulset-name --replicas number-of-replicas
where number-of-replicas is the desired number of replicated Pods.
Console
To scale a StatefulSet, perform the following steps:
Visit the Google Kubernetes Engine Workloads menu in Cloud Console.
In the workloads list, click the name of the StatefulSet you want to modify.
Click fullscreen Scale.
Enter the new number of Replicas for the StatefulSet.
Click Scale.
Deleting a StatefulSet
kubectl
To delete a StatefulSet, run the following command:
kubectl delete statefulset statefulset-name
Console
To delete a StatefulSet, perform the following steps:
Visit the Google Kubernetes Engine Workloads menu in Cloud Console.
In the workloads list, select one or more StatefulSets you want to delete.
Click delete Delete.
When prompted to confirm, click Delete.
What's next
- Learn about deploying stateless applications.
- Take a tutorial about upgrading a cluster running a stateful workload.