StatefulSet

This page describes Kubernetes StatefulSet objects and their use in Google Kubernetes Engine.

What is a StatefulSet?

StatefulSets represent a set of [Pods] with unique, persistent identities and stable hostnames that GKE maintains regardless of where they are scheduled. The state information and other resilient data for any given StatefulSet Pod is maintained in persistent disk storage associated with the StatefulSet.

StatefulSets use an ordinal index for the identity and ordering of their Pods. By default, StatefulSet Pods are deployed in sequential order and are terminated in reverse ordinal order. For example, a StatefulSet named web has its Pods named web-0, web-1, and web-2. When the web Pod specification is changed, its Pods are gracefully stopped and recreated in an ordered way; in this example, web-2 is terminated first, then web-1, and so on. Alternatively, you can specify the podManagementPolicy: Parallel field to have a StatefulSet launch or terminate all of its Pods in parallel, rather than waiting for Pods to become Running and Ready or to be terminated prior to launching or terminating another Pod.

StatefulSets use a Pod template, which contains a specification for its Pods. The Pod specification determines how each Pod should look: what applications should run inside its containers, which volumes it should mount, its labels and selectors, and more.

Usage patterns

StatefulSets are designed to deploy stateful applications and clustered applications that save data to persistent storage, such as Google Compute Engine persistent disks. StatefulSets are suitable for deploying Kafka, MySQL, Redis, ZooKeeper, and other applications needing unique, persistent identities and stable hostnames. For stateless applications, use Deployments.

Creating StatefulSets

You can create a StatefulSet using kubectl apply.

Once created, the StatefulSet ensures that the desired number of Pods are running and available at all times. The StatefulSet automatically replaces Pods that fail or are evicted from their nodes, and automatically associates new Pods with the storage resources, resource requests and limits, and other configurations defined in the StatefulSet's Pod specification.

The following is an example of a Service and StatefulSet manifest file:

apiVersion: v1
kind: Service
metadata:
  name: nginx
  labels:
    app: nginx
spec:
  ports:
  - port: 80
    name: web
  clusterIP: None
  selector:
    app: nginx
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: web
spec:
  selector:
    matchLabels:
      app: nginx # Label selector that determines which Pods belong to the StatefulSet
                 # Must match spec: template: metadata: labels
  serviceName: "nginx"
  replicas: 3
  template:
    metadata:
      labels:
        app: nginx # Pod template's label selector
    spec:
      terminationGracePeriodSeconds: 10
      containers:
      - name: nginx
        image: gcr.io/google_containers/nginx-slim:0.8
        ports:
        - containerPort: 80
          name: web
        volumeMounts:
        - name: www
          mountPath: /usr/share/nginx/html
  volumeClaimTemplates:
  - metadata:
      name: www
    spec:
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 1Gi

In this example:

  • A Service object named nginx is created, indicated by the metadata: name field. The Service targets an app called nginx, indicated by labels: app: nginx and selector: app: nginx. The Service exposes port 80 and names it web. This Service control the network domain and to route Internet traffic to the containerized application deployed by the StatefulSet.
  • A StatefulSet named web is created with three replicated Pods (replicas: 3).
  • The Pod template (spec: template) indicates that its Pods are labelled app: nginx.
  • The Pod specification (template: spec) indicates that the StatefulSet's Pods run one container, nginx, which runs the nginx-slim image at version 0.8. The container image is hosted by Container Registry.
  • The Pod specification uses the web port opened by the Service.
  • template: spec: volumeMounts specifies a mountPath, which is named www. The mountPath is the path within the container at which a storage volume should be mounted.
  • The StatefulSet provisions a PersistentVolumeClaim, www, with 1GB of provisioned storage.

In sum, the Pod specification contains the following instructions:

  • Label each Pod as app: nginx.
  • In each Pod, run one container named nginx.
  • Run the nginx-slim image at version 0.8.
  • Have Pods use port 80.
  • Save data to the mount path.

For more information about StatefulSet configurations, refer to the StatefulSet API reference.

Updating StatefulSets

You can update a StatefulSet by making changes to its Pod specification, which includes its container images and volumes. You can also update the object's resource requests and limits, labels, and annotations. To update a StatefulSet, you can use kubectl, the Kubernetes API, or the GKE Workloads menu in Google Cloud Console.

To decide how to handle updates, StatefulSets use a update strategy defined in spec: updateStrategy. There are two strategies, OnDelete and RollingUpdate:

  • OnDelete does not automatically delete and recreate Pods when the object's configuration is changed. Instead, you must manually delete the old Pods to cause the controller to create updated Pods.
  • RollingUpdate automatically deletes and recreates Pods when the object's configuration is changed. New Pods must be in Running and Ready states before their predecessors are deleted. With this strategy, changing the Pod specification automatically triggers a rollout. This is the default update strategy for StatefulSets.

StatefulSets update Pods in reverse ordinal order. You can monitor update rollouts by running the following command:

kubectl rollout status statefulset [STATEFULSET_NAME]

Partitioning rolling updates

You can partition rolling updates. Partitioning is useful if you want to stage an update, roll out a canary, or perform a phased roll out.

When you partition an update, all Pods with an ordinal greater than or equal to the partition value are updated when you update the StatefulSet’s Pod specification. Pods with an ordinal less than the partition value are not updated and, even if they are deleted, are recreated using the previous version of the specification. If the partition value is greater than the number of replicas, the updates are not propagated to the Pods.

What's next