Deploy ClickHouse on GCP GKE

This tutorial walks you through deploying ClickHouse Private on Google Cloud Platform using Google Kubernetes Engine (GKE), step by step. By the end, you will have a running ClickHouse cluster with GCS-backed storage, ephemeral local-SSD caching, and the ClickHouse operator managing the deployment.

For FIPS 140-3 / FedRAMP deployments, follow this guide for the GKE infrastructure, then apply the FIPS certificate and TLS configuration from Deploy with FIPS Compliance and Configure FIPS certificates.

For detailed infrastructure specifications, see reference/infrastructure-requirements.md. For an overview of how the operator works, see explanation/architecture.md.

Prerequisites

Before you begin, ensure you have the following tools installed:

gcloud CLI (gcloud) – installed and authenticated to your project
kubectl – compatible with your target GKE version
Helm v3.x
skopeo – for copying container images between registries
AWS CLI (aws) – with read access to the ClickHouse private ECR (<account-id-from-onboarding>.dkr.ecr.us-east-1.amazonaws.com; access details provided by ClickHouse during onboarding), used only for the image copy step

You will also need:

A GCP project with billing enabled and permissions to create GKE clusters, GCS buckets, IAM service accounts, and VPC resources
A bastion host in the VPC – the cluster is fully private, so all kubectl and Helm access must originate from within the VPC
The component versions for the current release, also listed on Latest Release:
- 26.2.1.525 – ClickHouse server image tag, to mirror into your registry
- 26.2.1.258 – ClickHouse keeper image tag, to mirror into your registry
- 1.20930.1 – Operator image and Helm chart tag
- 1.8.7 – Cluster Helm chart tag

Step 1: Create the Artifact Registry

Create a Docker-format Artifact Registry repository in the same region as your GKE cluster to hold the ClickHouse images and Helm charts, then configure Docker authentication against it.

GCP_PROJECT=your-project-id
GCP_REGION=us-central1
GAR_REPO=clickhouse

gcloud artifacts repositories create $GAR_REPO \
  --repository-format=docker \
  --location=$GCP_REGION \
  --description="ClickHouse container images" \
  --project=$GCP_PROJECT

gcloud auth configure-docker $GCP_REGION-docker.pkg.dev --quiet

The rest of this guide refers to the repository host as $GAR_HOST:

GAR_HOST=$GCP_REGION-docker.pkg.dev/$GCP_PROJECT/$GAR_REPO

Step 2: Copy Container Images

Use skopeo to copy images from the ClickHouse ECR into your Artifact Registry. The --all flag preserves all architectures (amd64, arm64).

SOURCE_ECR_ACCOUNT_ID=<account-id-from-onboarding>
SOURCE_ECR_REPO=$SOURCE_ECR_ACCOUNT_ID.dkr.ecr.us-east-1.amazonaws.com
GAR_HOST=$GCP_REGION-docker.pkg.dev/$GCP_PROJECT/$GAR_REPO

# log into the source ClickHouse ECR (requires AWS credentials provided by ClickHouse)
aws ecr get-login-password --region us-east-1 | skopeo login --username AWS --password-stdin $SOURCE_ECR_REPO

# log into your Artifact Registry
gcloud auth print-access-token | skopeo login --username oauth2accesstoken --password-stdin $GAR_HOST

# copy each image, be sure to include the --all flag
skopeo copy --all docker://$SOURCE_ECR_REPO/clickhouse-server:26.2.1.525 docker://$GAR_HOST/clickhouse-server:26.2.1.525
skopeo copy --all docker://$SOURCE_ECR_REPO/clickhouse-keeper:26.2.1.258 docker://$GAR_HOST/clickhouse-keeper:26.2.1.258
skopeo copy --all docker://$SOURCE_ECR_REPO/clickhouse-operator:main-1.20930.1 docker://$GAR_HOST/clickhouse-operator:main-1.20930.1
skopeo copy --all docker://$SOURCE_ECR_REPO/helm/clickhouse-operator-helm:1.20930.1 docker://$GAR_HOST/helm/clickhouse-operator-helm:1.20930.1
skopeo copy --all docker://$SOURCE_ECR_REPO/helm/onprem-clickhouse-cluster:1.8.7 docker://$GAR_HOST/helm/onprem-clickhouse-cluster:1.8.7

Verify the copy:

gcloud artifacts docker images list $GAR_HOST --format="table(package,version)"

Step 3: Create the VPC and Subnet

Create a custom-mode VPC and a GKE subnet in the deployment region. The subnet needs two secondary ranges – one for pods and one for services – and Private Google Access so private nodes can reach Google APIs (including GCS and Artifact Registry) without public IPs.

Setting	Value
VPC mode	Custom (not auto)
Subnet primary range	`10.1.0.0/24` (nodes)
Secondary range `pods`	`10.244.0.0/14`
Secondary range `services`	`10.252.0.0/20`
Control plane range	`172.16.0.0/28`
Bastion subnet	`10.0.0.0/24` (existing)
Private Google Access	Enabled
Cloud NAT	Required (see below)

CIDR boundary requirements

/14 blocks: the third octet must be divisible by 4 (e.g. 10.244.0.0, not 10.245.0.0)
/20 blocks: the fourth octet must be 0 and the third octet on a 16-boundary (e.g. 10.252.0.0)

For example, creating the subnet with both secondary ranges and Private Google Access:

gcloud compute networks subnets create gke-subnet \
  --network=clickhouse-vpc \
  --region=$GCP_REGION \
  --range=10.1.0.0/24 \
  --secondary-range=pods=10.244.0.0/14,services=10.252.0.0/20 \
  --enable-private-ip-google-access \
  --project=$GCP_PROJECT

Private nodes have no public IPs, so outbound internet access (for example, to pull the VolumeSnapshot CRDs in Step 7) requires Cloud NAT. Create a Cloud Router in the region and attach a Cloud NAT gateway covering all subnet ranges.

See reference/infrastructure-requirements.md for detailed networking requirements.

Step 4: Create the GKE Cluster

Create a fully private GKE cluster – worker nodes have only internal IPs, the control plane has no public endpoint, and all access must come from within the VPC via the bastion. Associate it with the VPC and subnet from Step 3, using the secondary ranges for pods and services and enabling Workload Identity.

Setting	Value	Purpose
`--enable-private-nodes`	Required	Nodes get internal IPs only
`--enable-private-endpoint`	Required	Control plane has no public IP (FedRAMP)
`--master-ipv4-cidr`	`172.16.0.0/28`	Control plane IP range
`--enable-master-authorized-networks`	Required	Enable network restrictions
`--master-authorized-networks`	GKE + bastion subnets	CIDRs allowed to reach the control plane
`--cluster-secondary-range-name`	`pods`	Pod IP alias range
`--services-secondary-range-name`	`services`	Service IP alias range
`--workload-pool`	`$GCP_PROJECT.svc.id.goog`	Enables Workload Identity

Once the cluster exists, fetch credentials from the bastion using the internal endpoint and confirm access:

GKE_CLUSTER_NAME=clickhouse-cluster

gcloud container clusters get-credentials $GKE_CLUSTER_NAME \
  --region=$GCP_REGION \
  --internal-ip \
  --project=$GCP_PROJECT

kubectl get nodes

Step 5: Create Node Pools

Create two node pools for ClickHouse, plus rely on the cluster’s default pool for the operator. Create one node pool per zone if you want the cluster autoscaler to balance across zones.

Keeper Node Pool

Setting	Value
Machine type	`n2-standard-8`
Disk	20 GB `pd-ssd` (node boot disk)
Keeper data volume	50Gi recommended; not part of the node pool, provisioned later by the Helm chart via the `keeper.storage` values (chart values reference)
Min/desired nodes	3 per ClickHouse cluster (if not autoscaling)
Workload metadata	`GKE_METADATA` (required for Workload Identity)
Kubernetes labels	`clickhouseGroup: keeper`
Kubernetes taints	`clickhouse.com/do-not-schedule: true, NoSchedule`

Server Node Pool

The server pool attaches its local NVMe SSD as ephemeral storage (--ephemeral-storage-local-ssd). GKE backs emptyDir volumes with that SSD, so the ClickHouse cache lands on NVMe automatically when the Helm chart sets server.ssdCacheConfiguration.isOnEmptyDir=true – which the GCP base configuration does by default (see Step 9). No node bootstrap script or DaemonSet is required.

Setting	Value
Machine type	`n2-standard-8` (dev) / `n2-standard-64` (prod)
Ephemeral local SSD	`--ephemeral-storage-local-ssd count=1`
Boot disk	20 GB `pd-standard`
Min/desired nodes	Equal to desired ClickHouse replicas (if not autoscaling)
Workload metadata	`GKE_METADATA` (required for Workload Identity)
Kubernetes labels	`clickhouseGroup: server`
Kubernetes taints	`clickhouse.com/do-not-schedule: true, NoSchedule`

Note - Scale local SSD count with the cache size

Each GKE local SSD is 375 GiB. When count is greater than 1, GKE combines the SSDs into a single RAID-0 (striped) volume and backs ephemeral storage with the combined capacity. count=1 suits the dev machine type; for larger servers, attach enough SSDs to hold the cache plus headroom for container images and logs. The cache size is bytesPerGiRAM * pod_memory_limit, so:

count = ceil( (bytesPerGiRAM * server_memory_GiB + headroom) / 375 GiB )

For example an n2-standard-64 (256 GiB RAM) with bytesPerGiRAM=9500Mi needs a cache of roughly 2.4 TiB, so count=8 (8 × 375 GiB = 3 TiB). Undersizing count fills the disk and the node runs out of ephemeral storage.

For example, creating the server pool with an ephemeral local SSD:

gcloud container node-pools create server-pool \
  --cluster=$GKE_CLUSTER_NAME \
  --region=$GCP_REGION \
  --machine-type=n2-standard-8 \
  --ephemeral-storage-local-ssd count=1 \
  --disk-type=pd-standard \
  --disk-size=20 \
  --num-nodes=1 \
  --enable-autoscaling --min-nodes=1 --max-nodes=10 \
  --workload-metadata=GKE_METADATA \
  --node-labels=clickhouseGroup=server \
  --node-taints=clickhouse.com/do-not-schedule=true:NoSchedule \
  --project=$GCP_PROJECT

Step 6: Create the GCS Bucket and Service Account

GCS Bucket

Create a Standard-class GCS bucket in the same region as the cluster, with uniform bucket-level access enabled. You can use one bucket per ClickHouse cluster, or a single bucket with a unique prefix per cluster.

Service Account and Workload Identity

ClickHouse authenticates to GCS and Artifact Registry through Workload Identity – there are no static credentials. Create a GCP service account (GSA), grant it access to the bucket and registry, then bind it to the Kubernetes service account (KSA) that the cluster Helm chart creates.

The chart creates a KSA named ch-$CLUSTER_NAME-sa in namespace ns-$CLUSTER_NAME. Because the name is deterministic, you can bind Workload Identity before deploying the cluster; the KSA annotation is applied by the chart at install time (Step 9), so pods have GCS access from first start.

GSA_NAME=clickhouse
GSA_EMAIL=$GSA_NAME@$GCP_PROJECT.iam.gserviceaccount.com
BUCKET_NAME=clickhouse-data-$GCP_PROJECT
CLUSTER_NAME=default-xx-01
NAMESPACE=ns-$CLUSTER_NAME

# create the service account
gcloud iam service-accounts create $GSA_NAME \
  --project=$GCP_PROJECT \
  --display-name="ClickHouse Workload Identity Service Account"

# grant bucket access
gcloud storage buckets add-iam-policy-binding gs://$BUCKET_NAME \
  --member=serviceAccount:$GSA_EMAIL --role=roles/storage.objectAdmin
gcloud storage buckets add-iam-policy-binding gs://$BUCKET_NAME \
  --member=serviceAccount:$GSA_EMAIL --role=roles/storage.legacyBucketReader

# grant Artifact Registry read access
gcloud artifacts repositories add-iam-policy-binding $GAR_REPO \
  --location=$GCP_REGION \
  --member=serviceAccount:$GSA_EMAIL \
  --role=roles/artifactregistry.reader \
  --project=$GCP_PROJECT

# bind Workload Identity to the chart-created KSA
gcloud iam service-accounts add-iam-policy-binding $GSA_EMAIL \
  --project=$GCP_PROJECT \
  --role=roles/iam.workloadIdentityUser \
  --member="serviceAccount:$GCP_PROJECT.svc.id.goog[$NAMESPACE/ch-$CLUSTER_NAME-sa]"

Workload Identity alignment:

Component	Value
GCP service account	`clickhouse@$GCP_PROJECT.iam.gserviceaccount.com`
K8s ServiceAccount (chart-created)	`ch-$CLUSTER_NAME-sa` in namespace `ns-$CLUSTER_NAME`
KSA annotation	`iam.gke.io/gcp-service-account=$GSA_EMAIL` (set by the chart in Step 9)
IAM binding member	`serviceAccount:$GCP_PROJECT.svc.id.goog[$NAMESPACE/ch-$CLUSTER_NAME-sa]`

Step 7: Install Kubernetes Prerequisites

Install VolumeSnapshot CRDs

These CRDs are required by the ClickHouse operator.

kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/master/client/config/crd/snapshot.storage.k8s.io_volumesnapshotclasses.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/master/client/config/crd/snapshot.storage.k8s.io_volumesnapshotcontents.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/master/client/config/crd/snapshot.storage.k8s.io_volumesnapshots.yaml

StorageClass

No StorageClass setup is required. GKE automatically creates the standard-rwo (Balanced persistent disk) and premium-rwo (SSD persistent disk) StorageClasses using the pre-installed GCE PD CSI driver (pd.csi.storage.gke.io). These are exactly the classes the GCP base configuration selects (server -> standard-rwo, keeper -> premium-rwo), and both use volumeBindingMode: WaitForFirstConsumer, so a disk is provisioned automatically in the zone where its pod is scheduled – no explicit topology configuration is needed.

Only create a custom StorageClass if you need non-default disk parameters, and override server.storage.storageClassName / keeper.storage.storageClassName accordingly.

Step 8: Install the Operator

Log into the Artifact Registry from Helm (note: no https:// prefix), then install the operator. Set the availability zones to your cluster’s zones.

GAR_HOST=$GCP_REGION-docker.pkg.dev/$GCP_PROJECT/$GAR_REPO

# operator Helm chart tag (eg 1.20930.1, not main-1.20930.1)
OPERATOR_VERSION=1.20930.1

# operator image tag -- the build itself (note the main- prefix)
OPERATOR_IMAGE_TAG=main-1.20930.1

# set zones as determined by the subnet
ZONES='["us-central1-a","us-central1-b","us-central1-c"]'

gcloud auth print-access-token | helm registry login \
  -u oauth2accesstoken --password-stdin $GCP_REGION-docker.pkg.dev

helm install clickhouse-operator \
   oci://$GAR_HOST/helm/clickhouse-operator-helm \
   --version=$OPERATOR_VERSION \
   --create-namespace \
   -n clickhouse-operator-system \
   --set-json="image.repository=\"$GAR_HOST/clickhouse-operator\"" \
   --set-json="image.tag=\"$OPERATOR_IMAGE_TAG\"" \
   --set-json='cilium.enabled=false' \
   --set-json='idleScalerEnabled=false' \
   --set-json='webhooks.enabled=false' \
   --set-json="operator.availabilityZones=$ZONES"

Step 9: Deploy a ClickHouse Cluster

Naming Your Cluster

Each ClickHouse cluster needs a unique name within the GKE cluster. Use the convention $DESCRIPTOR-$LETTERS-$ORDINAL:

$DESCRIPTOR – descriptive name using letters only
$LETTERS – reserved, use xx for simplicity
$ORDINAL – incrementing ordinal starting with 01
Example: default-xx-01

Generate Password Hash and Deploy

# this will be the `default` user's password.
# for production, pull it from Secret Manager instead, e.g.:
#   PASSWORD=$(gcloud secrets versions access latest --secret=clickhouse-password --project=$GCP_PROJECT)
PASSWORD='My super secret p@$$w0rd'
if command -v sha256sum &> /dev/null; then
  HASHED_PASSWORD=$(echo -n "$PASSWORD" | sha256sum | awk '{printf $1}' | base64 | tr -d '\n')
else
  HASHED_PASSWORD=$(echo -n "$PASSWORD" | shasum -a 256 | awk '{printf $1}' | base64 | tr -d '\n')
fi

# update values below as needed
CLUSTER_NAME=default-xx-01
GAR_HOST=$GCP_REGION-docker.pkg.dev/$GCP_PROJECT/$GAR_REPO
GSA_EMAIL=clickhouse@$GCP_PROJECT.iam.gserviceaccount.com
BUCKET_NAME=clickhouse-data-$GCP_PROJECT

# s3 key prefix can use any UUID value, but must be unique for all clusters storing data in the bucket
S3_KEY_PREFIX=ch-s3-$(uuidgen | tr '[:upper:]' '[:lower:]')

# these values should change depending on selected instance sizes.
# be sure to take DaemonSet requirements into account when setting CPU and MEMORY values
SERVER_CPU=4
SERVER_MEMORY=16Gi
# Keeper: high-load profile; for moderate load 2-4 CPU / 8Gi also works (see the infrastructure reference)
KEEPER_CPU=4
KEEPER_MEMORY=16Gi

# bytesPerGiRAM is a scaling factor used to automatically calculate the disk cache size.
# As a general rule, set cache size to 70-80% of the allocatable SSD disk, accounting
# for DaemonSets and cloud-provider reserved disk space. At pod startup:
#
# CONFIG_DISK_CACHE_SIZE = bytesPerGiRAM * pod_memory_limit
CACHE_BYTES_PER_GI_RAM=11Gi

CHART_VERSION=1.8.7

gcloud auth print-access-token | helm registry login \
  -u oauth2accesstoken --password-stdin $GCP_REGION-docker.pkg.dev

helm install $CLUSTER_NAME \
    oci://$GAR_HOST/helm/onprem-clickhouse-cluster \
    --version=$CHART_VERSION \
    -n ns-$CLUSTER_NAME \
    --create-namespace \
    --set-json='baseConfiguration.cloud="gcp"' \
    --set-json="account.hashedPassword=\"$HASHED_PASSWORD\"" \
    --set-json="serviceAccount.annotations={\"iam.gke.io/gcp-service-account\":\"$GSA_EMAIL\"}" \
    --set-json="server.image.repository=\"$GAR_HOST/clickhouse-server\"" \
    --set-json='server.arm64=false' \
    --set-json="server.storage.s3.bucketName=\"$BUCKET_NAME\"" \
    --set-json="server.storage.s3.keyPrefix=\"$S3_KEY_PREFIX\"" \
    --set-json="server.ssdCacheConfiguration.bytesPerGiRAM=\"$CACHE_BYTES_PER_GI_RAM\"" \
    --set-json="server.podPolicy.nodeSelector.clickhouseGroup=\"server\"" \
    --set-json="server.podPolicy.resources.limits.cpu=\"$SERVER_CPU\"" \
    --set-json="server.podPolicy.resources.limits.memory=\"$SERVER_MEMORY\"" \
    --set-json="server.podPolicy.resources.requests.cpu=\"$SERVER_CPU\"" \
    --set-json="server.podPolicy.resources.requests.memory=\"$SERVER_MEMORY\"" \
    --set-json='server.tolerations=[{"effect":"NoSchedule","key":"clickhouse.com/do-not-schedule","operator":"Exists"}]' \
    --set-json="keeper.image.repository=\"$GAR_HOST/clickhouse-keeper\"" \
    --set-json='keeper.arm64=false' \
    --set-json="keeper.podPolicy.nodeSelector.clickhouseGroup=\"keeper\"" \
    --set-json="keeper.podPolicy.resources.limits.cpu=\"$KEEPER_CPU\"" \
    --set-json="keeper.podPolicy.resources.limits.memory=\"$KEEPER_MEMORY\"" \
    --set-json="keeper.podPolicy.resources.requests.cpu=\"$KEEPER_CPU\"" \
    --set-json="keeper.podPolicy.resources.requests.memory=\"$KEEPER_MEMORY\"" \
    --set-json="keeper.storage.resources.requests=\"50Gi\"" \
    --set-json='keeper.tolerations=[{"effect":"NoSchedule","key":"clickhouse.com/do-not-schedule","operator":"Exists"}]'

Important GCP-specific settings:

baseConfiguration.cloud="gcp" – loads the GCP base configuration. This sets http_client=gcp_oauth on every server and keeper disk (so no per-disk overrides are needed), defaults the storage classes to standard-rwo (server) and premium-rwo (keeper), sets server.ssdCacheConfiguration.isOnEmptyDir=true so the cache uses the node’s ephemeral local SSD, and defaults object storage to the GCS S3-compatible endpoint https://storage.googleapis.com with region=auto (GCS does not use regions).
serviceAccount.annotations – annotates the chart-created KSA for Workload Identity. Combined with the IAM binding from Step 6, pods authenticate to GCS with no static credentials or useEnvironmentCredentials flag.
server.arm64=false / keeper.arm64=false – the node pools use x86 (n2) machine types. The chart defaults to arm64, so setting these false keeps the arm64-preferred labels and tolerations off the x86 pods.

Monitor the rollout (the operator creates server pods after keepers are healthy):

kubectl get pods -n ns-$CLUSTER_NAME -w

Step 10: Run Preflight Checks

To validate the readiness of your cluster we recommend running preflight checks. The preflight checks use Troubleshoot, a Kubernetes plugin for cluster diagnostics.

Install the Plugin

kubectl krew install preflight

Copy the Preflight Helm Chart

Add the preflight chart to your Artifact Registry:

skopeo copy --all \
  docker://$SOURCE_ECR_REPO/helm/preflight-check:1.38.4 \
  docker://$GAR_HOST/helm/preflight-check:1.38.4

Run the Checks

Use helm template to render the preflight spec, then pipe it to kubectl preflight:

CHART_VERSION=1.38.4
CLUSTER_NAME=default-xx-01

helm template clickhouse-preflight \
    oci://$GAR_HOST/helm/preflight-check \
    --version=$CHART_VERSION \
    --set preflight.cloud=gcp \
    --set preflight.clickhouseClusterName=$CLUSTER_NAME | \
kubectl preflight -

This validates node labels, StorageClass configuration, and other requirements. The output shows each check and its status. If a check fails, it includes recommendations on how to fix the issue.

For more details see the How To: Run Preflight Checks page.

Step 11: Verify Installation

Port-forward the ClickHouse Service

kubectl port-forward svc/c-default-xx-01-server-any 9000:9000 -n ns-default-xx-01

This forwards port 9000 to your local machine.

Connect and Run a Query

clickhouse client --host localhost --port 9000 --password $PASSWORD

Run a simple query:

SELECT 1;

Expected output:

   ┌─1─┐
1. │ 1 │
   └───┘
1 row in set. Elapsed: 0.001 sec.

Next Steps

FIPS / government compliance: See tutorials/deploy-government.md to apply FIPS 140-3 certificates and TLS configuration on top of this infrastructure.
Compute-Compute separation: See how-to/configure-compute-compute-separation.md to set up multiple compute groups with separate endpoints sharing a single dataset.
Management API: See tutorials/install-api.md to install the optional Private API for backups and scaling operations.
Monitoring and alerting: See how-to/configure-alerting.md to set up alerting for your deployment.
Troubleshooting: See troubleshooting.md for common issues and solutions.

Appendix: AWS to GCP Component Mapping

AWS Component	GCP Equivalent	Key Differences
ECR	Artifact Registry	Use `gcloud auth configure-docker`
S3	GCS with S3 API	Endpoint: `https://storage.googleapis.com`
IAM Role (IRSA)	Workload Identity	Annotate KSA with `iam.gke.io/gcp-service-account`
EBS CSI Driver	GCE PD CSI Driver	Pre-installed, provisioner: `pd.csi.storage.gke.io`
Availability Zones	GKE Zones	Topology key: `topology.kubernetes.io/zone`
NLB	GCP Load Balancer	Use `type: LoadBalancer` annotation