Stop reading whenever you have enough

mTLS, in twenty questions.

Each one is the question the previous answer makes you ask. Fourteen come with the manifest or the command, because that is the part you meet at work.

Not technical? The first three questions are written for you, and they are enough to follow any conversation about this.

Rungs
20
Hands on
14
Read
~12 min
01

What it is

01–03

Two programs talk over a wire. This is how each one proves who it is, instead of just saying so.

three minutes, no cluster needed · for anyone

  1. 01

    What problem is this solving?

    Not knowing this costs

    Without it, any program on the network can claim to be any service.

    • A service knows its caller only by the address on the message
    • The caller writes that address, so it can lie
    • mTLS replaces the claim with proof
  2. 02

    What does the m add to TLS?

    Not knowing this costs

    Most people assume plain TLS already does this. It does not.

    • Ordinary TLS: only the server proves who it is
    • That is a website proving itself to your browser
    • mTLS: the caller must present proof as well
  3. 03

    What do I have to build?

    Not knowing this costs

    This is the only reason mTLS ever gets adopted across a whole estate.

    • Nothing. No application change, in any language
    • A helper that runs beside each app, the proxy, does it all
    • You write policy, not code

Before you scroll on

0/3

You should now be able to

02

The YAML you will see

04–08

The rules are written down in small text files. These are the ones you will actually meet.

the five resources that appear in real repos · for whoever writes the manifests

  1. 04

    What actually switches it on?

    hands on

    Not knowing this costs

    No PeerAuthentication anywhere means PERMISSIVE, which accepts plaintext from anything.

    • PeerAuthentication, which controls what a server accepts
    • Name it default in istio-system to cover the whole mesh
    • Name it default in any namespace to cover that namespace
    manifestone namespace
    apiVersion: security.istio.io/v1
kind: PeerAuthentication
metadata:
  name: default        # the name matters
  namespace: payments  # scope is the namespace
spec:
  mtls:
    mode: STRICT       # refuse plaintext
    Put this in istio-system instead and it applies mesh wide. The name default is what makes it namespace scoped rather than workload scoped.
  2. 05

    How do I roll it out without an outage?

    hands on

    Not knowing this costs

    STRICT on day one cuts off every caller that has no proxy yet.

    • PERMISSIVE first: servers accept both plaintext and mTLS
    • Migrate callers one namespace at a time, watching for plaintext
    • STRICT last, once nothing plaintext remains
    manifeststep one, break nothing
    apiVersion: security.istio.io/v1
kind: PeerAuthentication
metadata:
  name: default
  namespace: payments
spec:
  mtls:
    mode: PERMISSIVE   # accept both while callers migrate
    Flip this same resource to STRICT as the final step. Going mesh wide, tighten namespace by namespace and leave istio-system for last.
  3. 06

    Do I need anything to make clients send mTLS?

    hands on

    Not knowing this costs

    Most mTLS incidents are a DestinationRule someone added for an unrelated reason.

    • Usually no. Auto mTLS is on by default
    • The proxy sends mTLS whenever the destination has a proxy
    • DestinationRule is only for overriding that
    manifestthe override that causes outages
    apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: payment-svc
  namespace: payments
spec:
  host: payment-svc.payments.svc.cluster.local
  trafficPolicy:
    tls:
      mode: ISTIO_MUTUAL   # DISABLE here is the classic outage
    Reserve DISABLE for genuinely non mesh endpoints. Against a STRICT server it resets every connection.
  4. 07

    How do I exempt one port, like a scrape endpoint?

    hands on

    Not knowing this costs

    The alternative people reach for is turning STRICT off for the whole workload.

    • portLevelMtls, on the same PeerAuthentication
    • Select the workload, then loosen only that port
    manifeststrict, except the metrics port
    spec:
  selector:
    matchLabels:
      app: legacy-exporter   # workload scoped, so no name: default
  mtls:
    mode: STRICT
  portLevelMtls:
    9090:
      mode: PERMISSIVE       # the one exception
    The port here is the container port, not the Service port. Getting that wrong exempts nothing and you will not be told.
  5. 08

    What is the identity, and where do I use it?

    hands on

    Not knowing this costs

    Two deployments sharing a service account are one identity, and no policy can separate them.

    • spiffe://cluster.local/ns/checkout/sa/checkout
    • Trust domain, namespace, Kubernetes service account
    • Best practice: deny everything first, then allow named callers
    manifestfirst, the default is no
    apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
  name: deny-all
  namespace: payments
spec: {}               # empty spec denies every request
    Apply this first. An allow list only means something when the default answer is no.
    manifestthen, only checkout may call payments
    apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
  name: payment-svc-callers
  namespace: payments
spec:
  selector:
    matchLabels:
      app: payment-svc
  action: ALLOW
  rules:
    - from:
        - source:
            principals:
              - cluster.local/ns/checkout/sa/checkout
    principals drops the spiffe:// prefix. Leaving it in matches nothing, and an ALLOW rule that matches nothing denies everyone.
fig 1Every combination of the two settings
client sendsauto (no DestinationRule)client sendsISTIO_MUTUALclient sendsDISABLE
serverSTRICT
works, encrypted
works, encrypted
every connection reset
serverPERMISSIVE
works, encrypted
works, encrypted
works, plaintext
serverDISABLE
works, plaintext
every connection reset
works, plaintext

Leave the client alone and only the bottom row is wrong. Add a DestinationRule with DISABLE and you buy two outages. This is why rung 6 says most incidents are a DestinationRule added for an unrelated reason.

Before you scroll on

0/3

You should now be able to

03

Proving it works

09–11

Trust nothing you cannot test. Three checks that end any argument about whether it is on.

commands that answer yes or no · for whoever has to sign it off

  1. 09

    Is mTLS on for this workload?

    hands on

    Not knowing this costs

    A manifest in git proves intent. Only this proves effect.

    • Ask the pod, not the manifest you applied
    • Mesh, namespace and workload policies all merge
    shellthe effective mode
    istioctl x describe pod httpbin-7d8b9c-4xk2 -n payments

# Effective PeerAuthentication:
#    Workload mTLS mode: STRICT
    PERMISSIVE here after you applied STRICT means your selector or namespace never matched this pod.
  2. 10

    How do I prove it with curl?

    hands on

    Not knowing this costs

    This is the one test that shows the identity that actually arrived.

    • Call a service that echoes headers, from inside the mesh
    • Look for X-Forwarded-Client-Cert in the response
    • That header only exists when the peer presented a certificate
    shellpositive test, from a pod with a proxy
    kubectl exec deploy/sleep -c sleep -n payments -- \
  curl -s http://httpbin:8000/headers

# "X-Forwarded-Client-Cert":
#   "By=spiffe://cluster.local/ns/payments/sa/httpbin;
#    Hash=8f2a...;
#    URI=spiffe://cluster.local/ns/payments/sa/sleep"
    URI is the caller. If that line is absent, the call arrived as plaintext no matter what your manifests say.
  3. 11

    How do I prove STRICT is really refusing plaintext?

    hands on

    Not knowing this costs

    Without this test you have only confirmed that mTLS is possible, not that plaintext is refused.

    • Call the same service from a pod with no proxy
    • Disable injection explicitly, or the test proves nothing
    • A refused connection is the pass condition
    shellnegative test, from a pod with no proxy
    kubectl run plain --image=curlimages/curl -n payments \
  --restart=Never --rm -it \
  --annotations=sidecar.istio.io/inject=false -- \
  curl -sS http://httpbin:8000/headers

# curl: (56) Recv failure: Connection reset by peer   <- STRICT works
# {"headers": {...}}                                  <- still PERMISSIVE
    Forget the annotation in an auto injected namespace and the pod gets a proxy, the call succeeds, and you conclude the wrong thing.

Before you scroll on

0/3

You should now be able to

04

When it breaks

12–14

The ways this fails all look mysterious from the outside. None of them are.

the failures you will actually hit · for whoever gets paged

  1. 12

    One dependency died and nothing else did. What is it?

    hands on

    Not knowing this costs

    Neither manifest looks wrong when you read it on its own.

    • Server on STRICT, client pinned to DISABLE
    • Flag UF with connection_termination, a few milliseconds
    • TLS error in the upstream proxy log
    shellconfirm it in two commands
    kubectl logs deploy/payment-svc -c istio-proxy -n payments --tail=20
# ...TLS error: 268435612:SSL routines:OPENSSL_internal:HTTP_REQUEST

kubectl get destinationrule -A -o custom-columns=\
NS:.metadata.namespace,NAME:.metadata.name,\
HOST:.spec.host,TLS:.spec.trafficPolicy.tls.mode
    HTTP_REQUEST in a TLS error means plaintext arrived on a port expecting mTLS. That is this failure, every time.
  2. 13

    I applied STRICT and nothing changed. Why?

    hands on

    Not knowing this costs

    A policy that applies to nothing looks identical in git to one that works.

    • A workload scoped policy needs a selector that matches
    • A namespace scoped policy must be named default
    • Traffic arriving from outside the mesh never sees it
    shellfind every policy that could apply
    kubectl get peerauthentication -A

# NAMESPACE      NAME      MODE
# istio-system   default   PERMISSIVE   <- mesh wide, and it wins nothing
# payments       strict    STRICT        <- not named default, so it needs
#                                            a selector to apply to anything
    Narrower scope always wins: workload beats namespace beats mesh. A workload policy with no matching selector applies to nothing at all.
  3. 14

    What happens when certificates expire?

    hands on

    Not knowing this costs

    An istiod outage is quiet for hours, then everything fails at once.

    • Workload certificates live 24 hours, renewed automatically
    • If istiod is down long enough, renewal stops
    • Then traffic stops, mesh wide, on its own
    shellread the expiry on a live pod
    istioctl proxy-config secret deploy/payment-svc -n payments \
  -o json | jq -r '.dynamicActiveSecrets[0]
    .secret.tlsCertificate.certificateChain.inlineBytes' \
  | base64 -d | openssl x509 -noout -enddate

# notAfter=Jul 27 09:14:02 2026 GMT
    Alert on istiod availability, not on this date. A healthy proxy renews at about half the lifetime and never gets near expiry.

Before you scroll on

0/3

You should now be able to

05

Beyond the basics

15–17

Once it works, three bigger questions arrive: whose certificates, how to watch it everywhere, and what happens at the edge.

the parts that appear at enterprise scale · for whoever owns the platform

  1. 15

    Should I keep the built in CA?

    hands on

    Not knowing this costs

    The default root is generated inside the cluster and dies with it.

    • For production, no. Bring your own root
    • Mount it as the cacerts secret before installing Istio
    • cert-manager with istio-csr automates its rotation
    shellyour root, not a generated one
    kubectl create secret generic cacerts -n istio-system \
  --from-file=ca-cert.pem \
  --from-file=ca-key.pem \
  --from-file=root-cert.pem \
  --from-file=cert-chain.pem

# install or restart istiod after this, then every new
# workload certificate chains to YOUR root
    Give istiod an intermediate, never the root key itself. Rotating this later means restarting workloads in waves, so plan it before go live rather than after.
  2. 16

    How do I see mTLS coverage across the whole mesh?

    hands on

    Not knowing this costs

    Per pod checks do not scale past a handful of services.

    • Every request metric carries a security label
    • connection_security_policy is mutual_tls or none
    • Kiali draws the same label as a padlock
    shellthe fleet answer, in one query
    # any Prometheus that scrapes Istio
sum by (connection_security_policy) (
  rate(istio_requests_total{reporter="destination"}[5m])
)

# connection_security_policy="mutual_tls"  <- the goal: all of it
# connection_security_policy="none"        <- plaintext still flowing
    Run this before every STRICT flip. The none series names exactly the traffic that is about to break, while it is still cheap to fix.
  3. 17

    What about traffic entering or leaving the mesh?

    hands on

    Not knowing this costs

    The padlock inside the mesh says nothing about the browser or the outside API.

    • Mesh mTLS covers proxy to proxy, inside the mesh
    • The browser to your gateway is separate TLS config
    • Leaving the mesh, the proxy can originate TLS for you
    manifestoriginate TLS on the way out
    apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: external-api
spec:
  host: api.stripe.com
  trafficPolicy:
    tls:
      mode: SIMPLE   # ordinary TLS, the outside has no mesh identity
    SIMPLE, not ISTIO_MUTUAL: external services hold no SPIFFE certificate. Pair it with a ServiceEntry so the mesh knows the host at all.

Before you scroll on

0/3

You should now be able to

06

Where it ends

18–20

What this does not protect you from, which is the part people get wrong in writing.

the limits, where people get caught · for whoever writes the compliance doc

  1. 18

    Is this end to end encryption?

    Not knowing this costs

    Do not write end to end in a compliance document. It is not true.

    • No. Encryption runs proxy to proxy
    • App to its own proxy is plaintext on loopback
  2. 19

    Does mTLS control who can call what?

    Not knowing this costs

    Perfect mTLS with no policies is still a completely flat network.

    • No. It answers who is calling, and stops there
    • Allowed or not is AuthorizationPolicy, from rung 8
  3. 20

    What does it not protect against?

    Not knowing this costs

    None of these three can be fixed by anything above.

    • A compromised pod that holds a valid identity
    • Anyone who can create pods in a namespace
    • Whoever controls the istiod root certificate
fig 2How far the encryption actually reaches

plaintext

loopback, inside the pod

encrypted

both ends prove identity

plaintext

loopback, inside the pod

your app
proxy
proxy
their app

all mTLS covers

Only the middle span is covered. The two ends are plaintext on loopback inside each pod, which is why end to end is the wrong phrase for this.

Not on this line at all: whether this caller was allowed to make this call. That is the AuthorizationPolicy from rung 8, and without it every proven identity may still reach every service.

Before you scroll on

0/3

You should now be able to

Go deeper

5 links, each earning its place.

Istio security concepts↗

The full model this page compresses: identity, certificates, and where PeerAuthentication and AuthorizationPolicy sit in it.

Mutual TLS migration task↗

The official runnable version of rung 5: PERMISSIVE to STRICT on a live cluster, step by step.

PeerAuthentication reference↗

Every field, including portLevelMtls, with the exact scoping rules rung 13 debugs.

Plugging in your own CA↗

The cacerts procedure from rung 15 in full, including the intermediate layout and verification.

SPIFFE, the identity standard↗

Why the identity string looks the way it does, and what federating trust domains actually means.

Where this leaves you

Rungs 1 to 3 are what most people need. Rungs 10 and 11 are the two commands that settle any argument about whether this is actually on.

If you keep one thing: mTLS answers who is calling. Whether they are allowed is a different resource, and a mesh that never wrote one is still flat.