Skip to main content
Career Paths
Concepts
Multi Cluster Istio
The Simplified Tech

Role-based learning paths to help you master cloud engineering with clarity and confidence.

Product

  • Career Paths
  • Interview Prep
  • Scenarios
  • AI Features
  • Cloud Comparison
  • Pricing

Community

  • Join Discord

Account

  • Dashboard
  • Credits
  • Updates
  • Sign in
  • Sign up
  • Contact Support

Stay updated

Get the latest learning tips and updates. No spam, ever.

Terms of ServicePrivacy Policy

© 2026 TheSimplifiedTech. All rights reserved.

BackBack
Interactive Explainer

Multi-Cluster Istio

How to span an Istio mesh across multiple Kubernetes clusters -- and the trust federation misconfiguration that breaks cross-cluster mTLS.

Relevant for:SeniorStaff
Why this matters at your level
Senior

Set up a basic multi-cluster mesh with east-west gateway. Know the shared root CA requirement and how to verify it. Debug cross-cluster mTLS failures.

Staff

Design multi-cluster mesh topology for multi-region HA. Own the PKI strategy across clusters. Define the upgrade procedure for multi-cluster Istio (control plane first, then data plane).

Multi-Cluster Istio

How to span an Istio mesh across multiple Kubernetes clusters -- and the trust federation misconfiguration that breaks cross-cluster mTLS.

~4 min read
Be the first to complete!
LIVEControl Plane Misconfiguration -- Trust Federation Failure -- Cross-Cluster mTLS Broken
Breaking News
T+0

Multi-cluster Istio configured -- primary (A) + remote (B)

T+1h

A -> B calls work. B -> A calls fail with TLS verification error

CRITICAL
T+2h

Investigation: cluster B using different root CA than cluster A

CRITICAL
T+3h

cacerts secret with shared root CA applied to cluster B -- B's istiod restarts

T+3.5h

All pods restarted to get new certs signed by shared CA -- cross-cluster mTLS works

—Cross-cluster B->A calls failing
—Time to diagnose and fix
—Missing cacerts secret in cluster B
—Different root CAs creating trust mismatch

The question this raises

How does multi-cluster Istio work, what trust model does it require, and how do you diagnose cross-cluster mTLS failures?

Test your assumption first

You set up multi-cluster Istio with clusters A and B. Services in A can call services in B successfully. Services in B cannot call services in A -- TLS verification error. Both clusters have east-west gateways. What is the most likely cause?

Lesson outline

Multi-cluster Istio topologies

Multi-cluster Istio is powerful but the trust model is the hard part

Multi-cluster Istio allows services in different Kubernetes clusters to call each other using mTLS as if they were in the same cluster. The traffic flows via east-west gateways between clusters. The security model requires all clusters to share the same root CA -- otherwise cross-cluster mTLS certificate verification fails.


Multi-primary topology (equal clusters, own istiod):
  Cluster A                          Cluster B
  +------------------+               +------------------+
  | istiod-A         |<-- sync ----->| istiod-B         |
  | (manages A)      |               | (manages B)      |
  +------------------+               +------------------+
  | East-West GW     |<-- mTLS ----->| East-West GW     |
  | (port 15443)     |               | (port 15443)     |
  +------------------+               +------------------+
  | Services A       |               | Services B       |
  +------------------+               +------------------+

Primary-remote topology (one istiod manages all):
  Cluster A (primary)                Cluster B (remote)
  +------------------+               +------------------+
  | istiod           |               | (no istiod)      |
  | (manages A+B)    |<- remote secret (kubeconfig for B)
  +------------------+               +------------------+
  | East-West GW     |<-- mTLS ----->| East-West GW     |
  +------------------+               +------------------+

TRUST REQUIREMENT (applies to all topologies):
  Both clusters MUST share the same root CA.
  Each cluster's workload SVIDs must be issued by the same root CA.
  If trust roots differ: destination Envoy cannot verify source cert = mTLS FAIL.

Shared root CA setup:
  1. Generate root CA (openssl or cloud KMS)
  2. Create cacerts secret in BOTH clusters:
     kubectl create secret generic cacerts \
       -n istio-system \
       --from-file=ca-cert.pem \
       --from-file=ca-key.pem \
       --from-file=root-cert.pem \
       --from-file=cert-chain.pem
  3. Install Istio AFTER cacerts exists (istiod uses it at startup)

Cross-cluster service discovery

How a service in cluster A finds and calls a service in cluster B

→

01

Service A in cluster A calls "service-b.namespace.svc.cluster.local"

→

02

Cluster A's Envoy looks up the service -- not found in cluster A's endpoint list

→

03

istiod in cluster A knows about cluster B services (via remote secret or mesh federation)

→

04

Envoy receives EDS entry for service-b in cluster B with the east-west gateway IP

→

05

Envoy routes to east-west gateway in cluster A (which forwards to cluster B)

→

06

East-west gateway in cluster A performs mTLS with east-west gateway in cluster B

→

07

Cluster B's east-west gateway forwards to service-b pod in cluster B

08

mTLS handshake verified using shared root CA -- SPIFFE identity validated cross-cluster

1

Service A in cluster A calls "service-b.namespace.svc.cluster.local"

2

Cluster A's Envoy looks up the service -- not found in cluster A's endpoint list

3

istiod in cluster A knows about cluster B services (via remote secret or mesh federation)

4

Envoy receives EDS entry for service-b in cluster B with the east-west gateway IP

5

Envoy routes to east-west gateway in cluster A (which forwards to cluster B)

6

East-west gateway in cluster A performs mTLS with east-west gateway in cluster B

7

Cluster B's east-west gateway forwards to service-b pod in cluster B

8

mTLS handshake verified using shared root CA -- SPIFFE identity validated cross-cluster

multi-cluster-setup.yaml
1# Step 1: Create shared root CA secret in BOTH clusters (before Istio install)
2# (Run this against each cluster)
3kubectl create secret generic cacerts -n istio-system \
4  --from-file=ca-cert.pem=./ca-cert.pem \
5  --from-file=ca-key.pem=./ca-key.pem \
6  --from-file=root-cert.pem=./root-cert.pem \
7  --from-file=cert-chain.pem=./ca-cert.pem
8 
9# Step 2: Install Istio with multi-cluster config (cluster A)
10# IstioOperator with meshID and clusterName
11apiVersion: install.istio.io/v1alpha1
12kind: IstioOperator
13metadata:
14  name: istio
15spec:
16  values:
17    global:
18      meshID: mesh1          # same for all clusters in mesh
19      clusterName: cluster-a  # unique per cluster
20      network: network-a     # network name (different if no direct pod connectivity)
21  components:
22    ingressGateways:
23    - name: istio-eastwestgateway
24      enabled: true
25      label:
26        istio: eastwestgateway
27      k8s:
28        service:
29          ports:
30          - port: 15443  # East-west traffic port
31            name: tls
kubectl
1# Verify east-west gateway is running in both clusters
2kubectl get pod -n istio-system -l istio=eastwestgateway
3 
4# Create remote secret (cluster A needs kubeconfig for cluster B)
5istioctl x create-remote-secret \
6  --context=cluster-b \
7  --name=cluster-b | kubectl apply -f - --context=cluster-a
8 
9# Check that cluster A knows about cluster B services
10istioctl remote-clusters --context=cluster-a
11# Should show: cluster-b: SYNCED
12 
13# Debug cross-cluster connectivity
14kubectl exec -n production my-pod -c istio-proxy -- \
15  curl -s http://service-b.production.svc.cluster.local/healthz
16# Should work if multi-cluster is configured correctly
17 
18# Check east-west gateway for cross-cluster requests
19kubectl logs -n istio-system -l istio=eastwestgateway \
20  -c istio-proxy | grep "cluster-b"
21 
22# Verify shared root CA in both clusters
23kubectl get secret cacerts -n istio-system --context=cluster-a \
24  -o jsonpath='{.data.root-cert\.pem}' | base64 -d | openssl x509 -noout -fingerprint
25kubectl get secret cacerts -n istio-system --context=cluster-b \
26  -o jsonpath='{.data.root-cert\.pem}' | base64 -d | openssl x509 -noout -fingerprint
27# Fingerprints must be identical for cross-cluster mTLS to work

What breaks in production

Blast radius of multi-cluster misconfiguration

  • Different root CAs — Cross-cluster mTLS fails -- destination cluster rejects source cluster certs
  • East-west gateway not exposed — No external IP on east-west gateway Service -- cross-cluster traffic has no entry point
  • Remote secret expired or wrong permissions — istiod cannot read remote cluster endpoints -- services not discovered cross-cluster
  • Network policies blocking east-west port — Port 15443 blocked between clusters -- all cross-cluster calls fail silently
  • Asymmetric network topology — Cluster A can reach cluster B IPs but B cannot reach A -- asymmetric routing breaks mTLS handshake

Installing Istio before creating the cacerts secret

Bug
# WRONG: Install Istio first, then try to add cacerts later
istioctl install --set values.global.meshID=mesh1

# Then try to add cacerts after install
kubectl create secret generic cacerts -n istio-system ...

# Problem: istiod already generated its own self-signed root CA at startup
# Restarting istiod after adding cacerts changes the CA
# All existing workload certs (signed by old self-signed CA) become invalid
# All pods must be restarted to get new certs -- service disruption
Fix
# RIGHT: Create cacerts BEFORE installing Istio
# Step 1: Create the cacerts secret
kubectl create namespace istio-system --context=cluster-a
kubectl create namespace istio-system --context=cluster-b

kubectl create secret generic cacerts -n istio-system \
  --context=cluster-a \
  --from-file=ca-cert.pem \
  --from-file=ca-key.pem \
  --from-file=root-cert.pem \
  --from-file=cert-chain.pem

# Same for cluster B
kubectl create secret generic cacerts -n istio-system \
  --context=cluster-b \
  --from-file=ca-cert.pem \
  --from-file=ca-key.pem \
  --from-file=root-cert.pem \
  --from-file=cert-chain.pem

# Step 2: Install Istio AFTER cacerts exists
# istiod will find cacerts and use it as the CA

Istio reads the cacerts secret at startup. If it is not present, istiod generates its own self-signed CA. Creating cacerts after Istio is installed requires restarting istiod AND restarting all pods to get new certs. In a multi-cluster context, this also means all cross-cluster trust is temporarily broken during the transition.

Decision guide: multi-cluster topology

Do all clusters have direct pod-to-pod IP connectivity (flat network)?
YesUse multi-primary with direct endpoint routing -- simpler, no east-west gateway needed for pod traffic
NoDo the clusters share the same cloud region and can the platform team maintain two istiod instances?
Do the clusters share the same cloud region and can the platform team maintain two istiod instances?
YesUse multi-primary with east-west gateway -- each cluster has its own istiod, more resilient, cross-cluster via gateway
NoUse primary-remote -- single istiod manages both clusters, simpler operationally but primary cluster is a control plane dependency for the remote

Multi-cluster topologies compared

Topologyistiod placementResilienceComplexityBest for
Single cluster1 istiod in clusterSingle cluster SLALowStarting point -- single region
Primary-remote1 istiod manages N clustersPrimary istiod is SPOF for remoteMediumDev/staging remote, tight control plane
Multi-primary1 istiod per clusterEach cluster independently operationalHighMulti-region HA, failure isolation
External control planeistiod outside all clustersDedicated control plane infraVery highManaged Istio services, large scale

Exam Answer vs. Production Reality

1 / 2

Multi-cluster trust model

📖 What the exam expects

All clusters in a multi-cluster Istio mesh must share the same root CA. Workload certificates from any cluster are signed by the same intermediate CA, allowing cross-cluster mTLS verification.

Toggle between what certifications teach and what production actually requires

How this might come up in interviews

Asked in senior platform and cloud architect interviews. "How would you achieve zero-downtime if one cloud region fails?" leads to multi-cluster Istio.

Common questions:

  • How does Istio route traffic between services in different clusters?
  • What trust model does multi-cluster Istio require?
  • What is the difference between primary-remote and multi-primary topology?
  • What is an east-west gateway and why does multi-cluster need it?
  • How would you debug a cross-cluster mTLS failure?

Strong answer: Mentions cacerts-before-install requirement, knows fingerprint verification step, has set up multi-cluster in practice, knows the latency overhead of east-west gateway hops.

Red flags: Does not know about the shared root CA requirement, thinks cross-cluster works automatically with just a kubeconfig, does not know about east-west gateway.

Related concepts

Explore topics that connect to this one.

  • Istio Architecture Deep Dive
  • Istio Advanced Routing
  • High availability (HA) & resilience

Suggested next

Often learned after this topic.

Istio Advanced Routing

Ready to see how this works in the cloud?

Switch to Career Paths for structured paths (e.g. Developer, DevOps) and provider-specific lessons.

View role-based paths

Discussion

Questions? Discuss in the community or start a thread below.

Join Discord

In-app Q&A

Sign in to start or join a thread.

Sign in to track your progress and mark lessons complete.

Continue learning

Istio Advanced Routing

Discussion

Questions? Discuss in the community or start a thread below.

Join Discord

In-app Q&A

Sign in to start or join a thread.