Infrastructure Requirements Reference

This document consolidates infrastructure prerequisites, node specifications, networking, storage, and IAM requirements across all supported ClickHouse Private deployment platforms.

Prerequisites by Platform

AWS (EKS)

GCP (GKE)

Azure (AKS)

Bare Metal

Node Group Specifications

Keeper quorum and failure domains

ClickHouse Keeper uses the Raft consensus algorithm. The Keeper ensemble can accept writes only while a majority of its replicas are available and can communicate with each other.

A standard ClickHouse Private Keeper ensemble uses three replicas. Use a different count only in consultation with ClickHouse Support.

For production HA, ensure that the three Keeper pods can run in three independent failure domains. ClickHouse Private spreads Keeper pods across the values of the node label configured by operator.topologyKey (default: topology.kubernetes.io/zone), with a maximum skew of one. Managed cloud Kubernetes services populate this default label from each node’s availability zone; provision Keeper capacity in three availability zones and verify the node labels before installation. On-premises, assign the label values yourself and map them to real fault-isolation boundaries such as separate racks, chassis, power feeds, or rooms. With only two domains, no placement can guarantee quorum after the loss of either domain.

Keeper sizing: node vs pod

The Keeper tables below describe two different things: the recommended instance type sizes the node, while the recommended resources size the Keeper pod. Each node runs a single Keeper pod, and nothing resizes the pod automatically, so size it for its expected peak rather than its starting point, with requests equal to limits as described in Pod QoS: Guaranteed. The resource rows list two load profiles: moderate (2-4 vCPU / 8Gi) and high (4 vCPU / 16Gi); high-rate insert workloads are high-load for Keeper. The recommended instance types are sized for the high profile; a moderate-load pod can run on a smaller node, subject to the allocatable-memory check below.

The pod cannot consume the node’s full memory capacity. Kubernetes schedules the Guaranteed Keeper pod by its memory request (equal to its limit) against the node’s allocatable memory, after subtracting the requests of DaemonSets and any other pods on that node. Use this check rather than the node’s advertised capacity:

kubectl get node <keeper-node> \
  -o custom-columns=NAME:.metadata.name,CAPACITY:.status.capacity.memory,ALLOCATABLE:.status.allocatable.memory
kubectl describe node <keeper-node>  # see "Allocated resources" for DaemonSet and other pod requests

As an initial hardware-planning allowance on common 16-64 GiB nodes, leave roughly 4-8 GiB for kubelet/system reservations, the eviction threshold, and node-level workloads, then verify the actual allocatable value and allocated requests for the chosen Kubernetes distribution and node shape. Reservations scale with node size and configuration, so 4-8 GiB is not a universal formula. The recommended 16Gi Keeper pod therefore needs a node with at least 24 GiB only if the checks above still leave 16 GiB allocatable to it; a 32 GiB node is the safer baseline and leaves room to raise the limit later. Sizing the node to exactly match the pod leaves no way to grow without replacing it.

When picking the pod’s memory figure, memory is the binding constraint: Keeper holds its entire dataset in memory, and resident usage grows with the logical metadata size. Track the ClickHouseAsyncMetrics_KeeperApproximateDataSize metric to watch that trend, and size memory from the expected metadata volume rather than a fixed guess.

A fixed allocation can be outgrown silently, so alert on Keeper memory headroom (Keeper sizing alerts has an example rule) and watch the growth of the Keeper metadata size. High-rate small inserts are the usual driver of metadata growth; batching inserts is the primary mitigation.

AWS (EKS)

Keeper Node Group

Server Node Group

Operator Node Group (x86)

All AWS nodes require IMDS for authentication.

For keeper and server node groups, create one node group per AZ if you wish to support cluster autoscaler across AZs.

Autoscaler tags (if using cluster-autoscaler):

• k8s.io/cluster-autoscaler/enabled: true
• k8s.io/cluster-autoscaler/$CLUSTER_NAME: owned

GCP (GKE)

Keeper Node Pool

Server Node Pool

Default Node Pool

Azure (AKS)

Keeper Node Pool

Server Node Pool

Temp storage by instance size (Standard_D*ds_v5, Intel Icelake):

System / Operator Node Pool

For keeper and server node pools, create one pool per availability zone if you wish to support cluster autoscaler across zones.

Bare Metal

Assign nodes to distinct roles and label and taint them so the operator schedules ClickHouse pods correctly and nothing else lands on them. For production HA, provide three independent failure domains, one for each Keeper node. The operator spreads Keeper pods across the node label configured by operator.topologyKey, which defaults to topology.kubernetes.io/zone, with a maximum skew of one. Map each domain to a real fault-isolation boundary such as a rack, chassis, or power feed.

Keeper Nodes

Server Nodes

Operator Nodes

At least one additional node for the operator and cluster add-ons. No ClickHouse labels or taints are required.

Pod QoS: Guaranteed (recommended)

ClickHouse server, keeper, and child compute group pods should run with the Kubernetes Guaranteed QoS class. In practice, this means that for every container in the pod, resources.requests equals resources.limits for both CPU and memory, and both fields are set. Every deployment example in this documentation follows this pattern.

Why

ClickHouse’s memory tracker relies on a stable memory ceiling. When requests != limits, the effective ceiling and the OOM-kill boundary diverge and behavior becomes hard to reason about:

• If the memory tracker uses the request value, queries are killed while the node still has headroom (over-conservative).
• If it uses the cgroup limit, the pod can exceed its request and be OOM-killed by the kernel with no ClickHouse-side warning.

Guaranteed QoS also gives the pod the highest priority under node pressure – see the Kubernetes Pod QoS docs for the eviction and scheduling implications.

Choosing a different QoS class

The operator does not enforce Guaranteed QoS – you can deliberately run ClickHouse under Burstable (or another class) if, for example, you are trading predictability for pack density. Before you do, make sure you understand the OOM trade-offs above and the wider Kubernetes resource model:

• Pod Quality of Service Classes
• Resource Management for Pods and Containers

For the applied examples, see any of the deployment tutorials or the compute-compute separation guide.

Networking Requirements

AWS

Example subnet layout (us-west-2):

GCP

Azure

Bare Metal

Storage Requirements

AWS

GCP

Azure

Bare Metal

IAM / Service Account Requirements

AWS

ECR Pull Role

• Role name: ClickHouseAirgapECRPullRole
• Purpose: Pull images from ClickHouse private ECR
• Permissions: ecr:GetDownloadUrlForLayer, ecr:BatchGetImage, ecr:BatchCheckLayerAvailability, ecr:DescribeImages, ecr:ListImages, ecr:GetAuthorizationToken
• Action: Provide the role ARN to the ClickHouse team

ClickHouse Server/Keeper S3 Role

• One role per provisioned ClickHouse cluster
• Naming convention: CH-S3-$NAME-$REGION-$ORDINAL-Role
  • $NAME: cluster name (e.g., default-xx-01)
  • $REGION: shortened region (e.g., uw2 for us-west-2)
  • $ORDINAL: reserved, set to 00
  • Example: CH-S3-default-xx-01-uw2-00-Role
• Permissions: s3:*, s3:ListBucket on the bucket resource
• Trust policy: IRSA via sts:AssumeRoleWithWebIdentity with EKS cluster’s OIDC provider

EBS CSI Driver Role

• Example name: ClickHouse_EksEbsCsiDriverRole
• Managed policy: arn:aws:iam::aws:policy/service-role/AmazonEBSCSIDriverPolicy
• Trust policy: IRSA with EKS cluster’s OIDC provider

GCP

Deployer Service Account

ClickHouse Workload Identity Service Account

• GCP SA: clickhouse@$PROJECT.iam.gserviceaccount.com
• K8s SA (manual): clickhouse in namespace ns-$CLUSTER_NAME
• K8s SA (helm-created): ch-$CLUSTER_NAME-sa in namespace ns-$CLUSTER_NAME
• K8s SA annotation: iam.gke.io/gcp-service-account=<GSA_EMAIL>
• IAM binding member: serviceAccount:$PROJECT.svc.id.goog[<namespace>/<ksa>]
• Bucket permissions: roles/storage.objectAdmin, roles/storage.legacyBucketReader
• Artifact Registry permissions: roles/artifactregistry.reader

Azure

Deployer Permissions

ClickHouse Managed Identity

• Type: User-Assigned Managed Identity (UAMI)
• K8s SA (helm-created): ch-$CLUSTER_NAME-sa in namespace ns-$CLUSTER_NAME
• K8s SA annotation: azure.workload.identity/client-id=<UAMI_CLIENT_ID>
• Federated credential subject: system:serviceaccount:ns-$CLUSTER_NAME:ch-$CLUSTER_NAME-sa
• Role assignment: Storage Blob Data Contributor on the Storage Account

Bare Metal

• Object storage credentials: Static access key / secret key for MinIO AIStor, held in a Kubernetes Secret (object-storage-credentials) and injected into server and keeper pods via additionalEnvVars.
• Registry pull secret: registry-credentials docker-registry secret in the clickhouse-operator-system and cluster namespaces, if your registry requires authentication.
• Service account: No cloud identity binding required.

Cluster Naming Conventions

The naming convention for ClickHouse clusters is consistent across all platforms:

$DESCRIPTOR-$LETTERS-$ORDINAL

Example name: default-xx-01

The cluster name must be unique within the target Kubernetes cluster and is used in:

• Kubernetes namespace: ns-$CLUSTER_NAME
• Kubernetes resources: prefixed with c-$CLUSTER_NAME
• IAM role names (AWS): CH-S3-$CLUSTER_NAME-$REGION-$ORDINAL-Role
• S3/GCS key prefixes: ch-s3-<uuid> (unique per cluster)

Required Kubernetes Components