Skip to content
ClickHouse Docs
ClickHouse DocsClickHouse Docs
49.6kGet Started

Backup Strategy for ClickHouse Private

View as MarkdownEdit this pageReport an issueSet up docs MCP

This document explains backup behavior and limitations specific to ClickHouse Private. For general ClickHouse backup concepts (full vs incremental, backup chains, SQL syntax), see the upstream ClickHouse backup documentation. For step-by-step procedures, see Back up and restore (manual) or Manage backups via API.

Backup Location

Backups are stored in Amazon S3. Each cluster’s backups are organized under the cluster’s S3 key prefix (found in the CR at spec.s3.keyPrefix), which ensures that backups from different clusters remain isolated even when sharing the same S3 bucket.

Manual vs API-Driven Backups

ClickHouse Private supports two backup approaches:

  • Manual (SQL) – Execute BACKUP/RESTORE statements directly on a server pod. Full control over scope and S3 paths. See Back up and restore (manual).
  • API-driven – Use the Private API REST endpoint, which creates Backup custom resources that the operator manages. See Manage backups via API and the API reference.

Backup Pod Designation

In the MultiSTS architecture, exactly one replica is designated as the backup pod (isBackupPod: true in the ReplicaStateMap). Backup operations always execute on this pod. The operator avoids marking the backup pod as Condemned during scale-in to prevent disrupting backup operations.

Restore Limitation

The Private API disallows restoring a backup to the same cluster it was taken from (instance_id == target_instance_id). This prevents placing additional load on a running production cluster and reduces the risk of data overwrites. The recommended approach is to provision a new cluster and restore into it.

If you must restore to the same cluster (for example, to recover a specific table), use manual SQL commands directly on a server pod.

Lifecycle management

There is no one size fits all approach to lifecycle management for backups. Sevaral strategies exist:

  1. Using Cloud storage lifecycle policies (i.e. Object Lifecycle Rules). Care should be taken to avoid breaking the backup chain (see the section below)
  2. Custom CronJob in Kubernetes that queries the backups & determines which backups can be safely removed.

Prepare your strategy in advance and align it with your objectives. Make sure to regularly test your backup & restore strategy by testing old backups & restoring them on a new ClickHouse cluster.

Incremental Backup Chain Integrity

When using S3 lifecycle policies with incremental backups, ensure that expiration rules do not delete a full backup while incremental backups that depend on it still exist. If a full backup is deleted, all incremental backups in its chain become unrestorable.

Navigation