Skip to content
ClickHouse Docs
ClickHouse DocsClickHouse Docs

Move the filesystem cache to a StorageClass

The filesystem cache on each Server replica can be backed by a PersistentVolumeClaim provisioned from a StorageClass. Any StorageClass works, whether it provisions node-local NVMe or network-attached storage, although node-local consistently gives the best cache performance.

Setting it up on a new instance is a normal install. Moving an instance that is already running onto a StorageClass, or off it again, takes an extra manual step, and this page covers that case.

Why the extra step is needed

A StorageClass-backed cache comes from the StatefulSet’s volumeClaimTemplates, and that field is immutable. The Operator cannot add it to, or remove it from, a StatefulSet that already exists. The StatefulSet has to be replaced instead.

Move onto a StorageClass

Set the StorageClass in your Helm values

server:
  ssdCacheConfiguration:
    cacheDiskSize: "400Gi"
    isOnEmptyDir: false
    storage:
      storageClassName: <your-storage-class>
      size: "400Gi"

accessModes and volumeMode default to ReadWriteOnce and Filesystem. The cache is per-replica and never shared, so these suit most drivers.

Size the claim at or above the cache budget you give ClickHouse. ClickHouse fills to that budget whatever the volume’s actual size, so an undersized claim fails writes with ENOSPC instead of evicting from the cache.

Upgrade the release. The replicas will start failing as described above.

Recreate each Server StatefulSet, one replica at a time

Each Server replica runs in its own StatefulSet with a single pod. List them:

kubectl get statefulsets -n <namespace> \
  -l app.kubernetes.io/name=clickhouse-server,app=<instance-name>-server

Delete one and wait for the Operator to rebuild it:

kubectl delete statefulset <sts-name> -n <namespace>
kubectl rollout status statefulset/<sts-name> -n <namespace>

The Operator recreates it from scratch, this time with the cache claim template in place, which takes a couple of minutes. Confirm the replica is healthy before moving on to the next one.

Verify

The recreated StatefulSet should list both claim templates:

kubectl get statefulset <sts-name> -n <namespace> \
  -o jsonpath='{range .spec.volumeClaimTemplates[*]}{.metadata.name}{"\n"}{end}'
# ch-storage-volume
# ch-cache-volume

And each replica should have a bound cache claim:

kubectl get pvc -n <namespace> -l app=<instance-name>-server

Once every replica has been recreated the instance returns to Running:

kubectl get clickhousecluster -n <namespace> <instance-name>

Move back off a StorageClass

The same constraint applies in reverse. Once a StatefulSet carries a cache claim template the Operator cannot remove it, so clearing storage.storageClassName and choosing another backing is not complete until each StatefulSet has been recreated. Follow the same procedure.

The old ch-cache-volume claims stay bound afterwards, because the Operator only collects cache volumes whose replica has gone away. Delete them once every replica is healthy to release the capacity:

kubectl delete pvc -n <namespace> ch-cache-volume-<sts-name>-0