Lesson 8 of 11 · 10 min · Cluster networking

Cluster DNS and network policies

Covers: Kubernetes 1.34 (k3s with built-in network policy enforcement), kubectl 1.34. Updated 13 Nov 2026. Part of the DevOps Engineer roadmap.

What you will learn

  • How Kubernetes DNS works and what names exist
  • The Pod's resolv.conf, the search list and ndots:5
  • Headless Services and ExternalName in DNS
  • NetworkPolicies: selecting Pods, allowing traffic, default deny
  • The mistakes that break DNS or lock everything out

In simple words

Inside a cluster, a Pod calls a Service by name, and the cluster's own phone book (DNS) turns that name into an address. By default every Pod may talk to every other Pod. Network policies are the rules that say who may talk to whom, like visitor rules inside a building.

Words to know

  • Cluster DNS: the name lookup service inside Kubernetes.
  • Service name: what you call to reach a Service, such as db or db.shop.svc.cluster.local.
  • NetworkPolicy: a rule that allows or blocks traffic between Pods.
  • Default deny: blocking everything first, then allowing only what is needed.

Cluster DNS

Every cluster runs a DNS service (CoreDNS, in kube-system, itself a Deployment behind the Service kube-dns). The kubelet configures each Pod to use it. Inside a Pod:

kubectl exec dnst -- cat /etc/resolv.conf
search default.svc.cluster.local svc.cluster.local cluster.local
nameserver 10.43.0.10
options ndots:5

10.43.0.10 is the ClusterIP of the kube-dns Service. CoreDNS watches the Kubernetes API and answers for the cluster domain (cluster.local) from live data, and forwards other names to upstream resolvers.

Names you get

Object DNS name Resolves to
Service <svc>.<namespace>.svc.cluster.local the ClusterIP
Headless Service same name the IPs of the individual ready Pods
StatefulSet Pod <pod>.<svc>.<namespace>.svc.cluster.local that Pod's IP (stable names, db-0.db)
Pod <ip-with-dashes>.<namespace>.pod.cluster.local the Pod IP (rarely useful)
ExternalName Service the Service name a CNAME to the external name

Our lab:

api-ci.default.svc.cluster.local     -> 10.43.188.171           (ClusterIP)
api-headless.default.svc.cluster.local -> 10.42.0.9, 10.42.0.8  (the two Pod IPs)
ext-db   -> canonical name = example.com                       (ExternalName)

The search list and ndots

Because of the search line, short names work. From a Pod in namespace default:

  • api-ci → tried as api-ci.default.svc.cluster.local ✓.
  • api-ci.team-a (a Service in another namespace) → api-ci.team-a.svc.cluster.local ✓.
  • a name in another namespace without qualification fails (NXDOMAIN), as in the namespaces lesson.

ndots:5 means: a name with fewer than five dots is first tried with each search suffix appended, and only afterwards as-is. So looking up example.com (one dot) first tries example.com.default.svc.cluster.local, example.com.svc.cluster.local, example.com.cluster.local, each a failed query, before the real one. For applications that call many external hosts this multiplies DNS traffic and latency. Fixes: write external names with a trailing dot (example.com.), lower ndots in the Pod's dnsConfig, or cache locally (NodeLocal DNSCache). Do not confuse this with a "DNS problem": the lookups are working as designed.

Check the DNS configuration: kubectl -n kube-system get cm coredns -o yaml shows the Corefile with the kubernetes cluster.local plugin and a forward line for outside names. DNS Pods run on port 53 (UDP and TCP); a NetworkPolicy that blocks egress to them breaks everything (see below).

Debugging DNS

  1. kubectl run dnst --image=busybox:1.36 --restart=Never --command -- sleep 600 (a tool Pod), then kubectl exec dnst -- nslookup <name>.
  2. Name not found? Check namespace and Service name (kubectl get svc -A | grep ...).
  3. Times out? Check that CoreDNS Pods are running (kubectl -n kube-system get pods -l k8s-app=kube-dns), view their logs, and look for NetworkPolicies blocking port 53.
  4. Slow lookups? Think ndots and upstream resolver latency.
  5. Application resolves but nslookup doesn't (or vice versa)? Applications and caches differ (Docker/Networking courses).

Network policies

By default every Pod can talk to every other Pod in the cluster (and to the outside). That is convenient and dangerous: a compromised front end can reach the database directly. A NetworkPolicy is a firewall rule for Pods, written with labels. Important facts:

  • Policies are enforced by the network plugin (CNI), not by Kubernetes itself. Calico, Cilium and Antrea enforce them; some simple plugins (plain flannel) do not, and a policy would silently do nothing. k3s bundles an enforcer, which is why our test worked. Verify enforcement on your cluster with a test like the one below.
  • Policies are additive allow-lists. A Pod that is selected by no policy accepts all traffic. Once a Pod is selected by a policy for a direction (ingress/egress), only traffic matched by some policy for that direction is allowed; everything else is dropped.

Our experiment: a db Deployment and Service in namespace shop, and two client Pods labelled role=api and role=other.

Before any policy both clients reached the database:

before policy: allowed=<!DOCTYPE html>  intruder=<!DOCTYPE html>

Policy 1: allow ingress to db only from Pods labelled role=api:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata: {name: db-allow-api, namespace: shop}
spec:
  podSelector: {matchLabels: {app: db}}      # the policy applies to the db Pods
  policyTypes: [Ingress]
  ingress:
    - from:
        - podSelector: {matchLabels: {role: api}}
      ports: [{port: 80}]

After:

after policy:  allowed=<!DOCTYPE html>  intruder=wget: can't connect to remote host (10.43.250.171)

The unlabelled client is now blocked, and the role=api one still works.

How to read a policy:

  • podSelector: which Pods the policy protects (empty {} = all Pods in the namespace).
  • policyTypes: Ingress (traffic into those Pods), Egress (traffic out), or both.
  • from / to entries can use podSelector (Pods in the same namespace), namespaceSelector (all Pods in matching namespaces; combine with podSelector in the same list item for AND), and ipBlock (CIDR ranges, for outside addresses).
  • ports restrict ports and protocols.
  • A common mistake is mixing up list items: two entries under from are OR; selectors inside one entry are AND.

Default deny

The recommended baseline is to deny everything in a namespace, then allow what is needed:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata: {name: default-deny-all, namespace: shop}
spec:
  podSelector: {}
  policyTypes: [Ingress, Egress]

Then we tested the allowed client again:

allowed after default-deny: wget: bad address 'db'
allowed can resolve DNS?  ;; connection timed out; no servers could be reached

Everything stopped, including DNS: egress to CoreDNS (port 53 UDP/TCP in kube-system) is also denied, so even the name db cannot be resolved. A real default-deny setup therefore always adds an egress rule allowing DNS:

egress:
  - to:
      - namespaceSelector: {matchLabels: {kubernetes.io/metadata.name: kube-system}}
    ports:
      - {port: 53, protocol: UDP}
      - {port: 53, protocol: TCP}

plus rules for the specific services each application calls. Build policies per application tier (front end → API → database), keep them in Git next to the application manifests, and test them: a policy that was never tested may not do what you think. Tools such as Cilium's Hubble or Calico flow logs show which flows are dropped.

What network policies do not cover

They work at L3/L4 (IPs, ports) and say nothing about HTTP paths or identities; a service mesh adds mTLS and L7 authorisation. They also do not protect the node or control-plane components, and they are not a substitute for RBAC and Pod security controls.

Common mistakes

  • Assuming policies are enforced when the CNI does not support them.
  • Default deny without allowing DNS (everything "breaks").
  • AND vs OR confusion in from lists.
  • Forgetting both directions: ingress to the destination and egress from the source if egress is restricted.
  • Labels that don't match (policy does nothing).
  • Thinking ndots lookups are failures.
  • Hard-coding Pod IPs instead of Service names.

Try it yourself

  1. In a Pod, run nslookup for a Service, a headless Service and an ExternalName Service, and read /etc/resolv.conf.
  2. Reproduce the allow/deny test with two labelled client Pods and a database Pod.
  3. Add default deny, observe DNS and everything else fail, then add the DNS egress rule and one allow rule to restore the legitimate path.
  4. Measure how many DNS queries curl example.com causes inside a Pod (watch the CoreDNS logs or use tcpdump) and reduce it with a trailing dot.

Check yourself

What is the full DNS name of Service api in namespace team-a?

api.team-a.svc.cluster.local. From within team-a, plain api works; from other namespaces use api.team-a.

What does ndots:5 do?

Names with fewer than five dots are first tried with each search-domain suffix, so external names like example.com trigger several failing lookups before the real one. Use a trailing dot or tune dnsConfig to reduce it.

By default can any Pod reach any other Pod?

Yes. NetworkPolicies, enforced by the network plugin, are how you restrict that.

After applying a default-deny policy, why did name resolution fail?

Egress to CoreDNS was blocked too. Allow egress to the DNS Pods on port 53 (UDP and TCP).

Interview questions

How would you restrict database access to the API tier only?

A NetworkPolicy on the database Pods allowing ingress only from Pods labelled as the API on the database port, on top of a default deny for the namespace, with the API's egress allowed to the database and DNS. Verify enforcement with a negative test.

A Service name does not resolve from a Pod. What do you check?

The name and namespace; CoreDNS health and logs; the Pod's resolv.conf; whether a policy blocks DNS; and whether the Service actually exists (kubectl get svc -A).

What is a headless Service used for in DNS?

Returning the individual Pod IPs (and stable per-Pod names for StatefulSets) instead of a single virtual IP.

Summary

  • CoreDNS gives Services, headless Services and StatefulSet Pods stable names under cluster.local; the search list makes short names work and ndots:5 multiplies external lookups.
  • Pods can talk freely by default; NetworkPolicies (enforced by the CNI) allow only listed traffic once a Pod is selected.
  • Start from default deny, allow DNS, then allow each required flow; test enforcement.

Going deeper

See how much of this you can explain

A free mock interview asks you new questions on Kubernetes for DevOps Engineer and scores every answer, with what to fix.