Your pods can't reach other services by name. DNS lookups fail. nslookup returns errors. CoreDNS is the internal DNS server for Kubernetes, and when it's broken, your cluster can't function properly.
DNS resolution failures affect all service-to-service communication. Let's diagnose CoreDNS and fix the problem.
Introduction
This article covers troubleshooting steps and solutions for Fix Kubernetes CoreDNS Not Resolving. The error typically occurs in production environments and can cause service disruptions if not addressed promptly.
Symptoms
Common error messages include:
kubectl run dns-test --image=busybox:1.28 --rm -it --restart=Never -- sh```bash # Test DNS resolution nslookup kubernetes.default
# Expected output: Server: 10.96.0.10 Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local
Name: kubernetes.default Address 1: 10.96.0.1 kubernetes.default.svc.cluster.local ```
``` Server: 10.96.0.10 Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local
nslookup: can't resolve 'kubernetes.default': Name does not resolve ```
Common Causes
- Configuration misconfiguration
- Missing or incorrect credentials
- Network connectivity issues
- Version compatibility problems
- Resource exhaustion or limits
- Permission or access denied
Step-by-Step Fix
- 1.Check logs for specific error messages
- 2.Verify configuration settings
- 3.Test network connectivity
- 4.Review recent changes
- 5.Apply corrective action
- 6.Verify the fix
Understanding Kubernetes DNS
Kubernetes uses CoreDNS (or kube-dns in older clusters) to provide DNS resolution for:
- Service names: my-service
- Service with namespace: my-service.production
- Full FQDN: my-service.production.svc.cluster.local
- External DNS: external.example.com
Each pod gets DNS configuration pointing to the cluster DNS service.
Step 1: Verify DNS Resolution Failure
From a test pod, verify DNS is failing:
kubectl run dns-test --image=busybox:1.28 --rm -it --restart=Never -- shInside the pod:
```bash # Test DNS resolution nslookup kubernetes.default
# Expected output: Server: 10.96.0.10 Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local
Name: kubernetes.default Address 1: 10.96.0.1 kubernetes.default.svc.cluster.local ```
If it fails:
``` Server: 10.96.0.10 Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local
nslookup: can't resolve 'kubernetes.default': Name does not resolve ```
Step 2: Check CoreDNS Pods
Verify CoreDNS pods are running:
kubectl get pods -n kube-system -l k8s-app=kube-dnsNAME READY STATUS RESTARTS AGE
coredns-5d7b9c8f7-x9k2m 1/1 Running 0 30d
coredns-5d7b9c8f7-p4j3n 1/1 Running 0 30dIf pods are not running:
NAME READY STATUS RESTARTS AGE
coredns-5d7b9c8f7-x9k2m 0/1 CrashLoopBackOff 5 5mCheck pod logs:
kubectl logs -n kube-system coredns-5d7b9c8f7-x9k2mLook for errors like: - Configuration errors - Upstream DNS failures - Memory/resource limits
Step 3: Check CoreDNS Service
Verify the DNS service exists and has endpoints:
kubectl get svc -n kube-system kube-dnsNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kube-dns ClusterIP 10.96.0.10 <none> 53/UDP,53/TCP 30dCheck endpoints:
kubectl get endpoints -n kube-system kube-dnsNAME ENDPOINTS AGE
kube-dns 10.244.0.5:53,10.244.0.6:53 30dIf endpoints are empty, CoreDNS pods aren't ready or the service selector is wrong.
Verify service selector matches pod labels:
```bash kubectl get svc kube-dns -n kube-system -o jsonpath='{.spec.selector}' # {"k8s-app":"kube-dns"}
kubectl get pods -n kube-system -l k8s-app=kube-dns ```
Step 4: Check Pod DNS Configuration
Verify pods are configured to use CoreDNS:
kubectl exec dns-test -- cat /etc/resolv.confnameserver 10.96.0.10
search default.svc.cluster.local svc.cluster.local cluster.local
options ndots:5The nameserver should match the kube-dns ClusterIP.
If the nameserver is wrong (e.g., 8.8.8.8), check kubelet DNS configuration:
# On a node
cat /var/lib/kubelet/config.yaml | grep -A 5 dnsCorrect configuration:
clusterDNS:
- 10.96.0.10
clusterDomain: cluster.localStep 5: Test Direct CoreDNS Connectivity
Test if you can reach CoreDNS directly:
kubectl exec dns-test -- nc -zv 10.96.0.10 53If connection fails, there's a network issue between pods and CoreDNS.
Test from CoreDNS pod itself:
kubectl exec -n kube-system coredns-5d7b9c8f7-x9k2m -- nslookup kubernetes.defaultIf this works but external pods fail, there's a network policy or connectivity issue.
Step 6: Check CoreDNS Configuration
View CoreDNS Corefile:
kubectl get configmap coredns -n kube-system -o yamlapiVersion: v1
data:
Corefile: |
.:53 {
errors
health {
lameduck 5s
}
ready
kubernetes cluster.local in-addr.arpa ip6.arpa {
pods insecure
fallthrough in-addr.arpa ip6.arpa
ttl 30
}
prometheus :9153
forward . /etc/resolv.conf {
max_concurrent 1000
}
cache 30
loop
reload
loadbalance
}
kind: ConfigMapCommon issues in Corefile:
- Missing kubernetes plugin
- Wrong cluster.local domain
- Forward DNS not working
Step 7: Check Upstream DNS Forwarding
CoreDNS forwards external DNS queries to upstream servers. Test if upstream DNS works:
# From CoreDNS pod, check upstream config
kubectl exec -n kube-system coredns-5d7b9c8f7-x9k2m -- cat /etc/resolv.confIf upstream DNS servers are unreachable, external DNS resolution fails.
Test upstream DNS:
kubectl exec dns-test -- nslookup google.com 8.8.8.8If this fails, the issue is with upstream DNS, not CoreDNS.
Step 8: Check Network Policies
Network policies can block DNS traffic:
kubectl get networkpolicy -n kube-system
kubectl get networkpolicy -n defaultDNS uses UDP port 53. Check if policies allow DNS:
# Example policy that might block DNS
spec:
podSelector: {}
policyTypes:
- Egress
egress:
- to:
- namespaceSelector: {} # Missing kube-system namespace
ports:
- protocol: UDP
port: 53Fix by allowing kube-system namespace:
- to:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: kube-system
ports:
- protocol: UDP
port: 53Step 9: Check CoreDNS Resource Limits
CoreDNS might be resource-starved:
kubectl describe pod -n kube-system -l k8s-app=kube-dns | grep -A 5 "Limits:"Limits:
cpu: 100m
memory: 70MiIf limits are too low, CoreDNS can't handle query load. Increase:
kubectl patch deployment coredns -n kube-system --type=merge -p '{"spec":{"template":{"spec":{"containers":[{"name":"coredns","resources":{"limits":{"cpu":"200m","memory":"150Mi"}}}]}}}}'Step 10: Check DNS Cache and Loop Issues
DNS loop errors occur when CoreDNS forwards to itself:
kubectl logs -n kube-system -l k8s-app=kube-dns | grep -i loopIf you see:
[ERROR] plugin/errors: Forward loop detectedThis happens when node's /etc/resolv.conf points to CoreDNS. Fix:
```bash # On nodes, change upstream DNS cat /etc/resolv.conf # If it contains 10.96.0.10 (cluster DNS), change to external DNS
# Fix kubelet upstream DNS echo "nameserver 8.8.8.8" > /etc/resolv.conf ```
Common CoreDNS Issues
Issue: CoreDNS pods not starting
coredns CrashLoopBackOffCause: RBAC issues or missing service account.
Solution:
kubectl get serviceaccount coredns -n kube-system
kubectl get clusterrolebinding corednsIf missing, recreate:
kubectl apply -f https://raw.githubusercontent.com/kubernetes/kubernetes/master/cluster/addons/dns/coredns/coredns.yamlIssue: DNS resolution slow or intermittent
Cause: Resource limits too low for query volume.
Solution: Increase resources and replicas:
kubectl scale deployment coredns -n kube-system --replicas=3
kubectl patch deployment coredns -n kube-system --type=merge -p '{"spec":{"template":{"spec":{"containers":[{"name":"coredns","resources":{"limits":{"memory":"200Mi"}}}]}}}}'Issue: External DNS not resolving
nslookup: can't resolve 'google.com'Cause: Upstream DNS forwarding broken.
Solution: Update Corefile forward directive:
forward . 8.8.8.8 8.8.4.4 {
max_concurrent 1000
}Apply:
kubectl edit configmap coredns -n kube-systemIssue: Custom domain not resolving
Cause: Corefile missing stub domain configuration.
Solution: Add stub domain:
Corefile: |
example.com:53 {
forward . 192.168.1.1
cache 30
}
.:53 {
# existing config
}Issue: DNS queries dropped on high load
Cause: UDP buffer size too small.
Solution: Increase buffer in CoreDNS:
forward . /etc/resolv.conf {
max_concurrent 1000
policy sequential
}Verification
After fixing CoreDNS:
```bash # Test DNS resolution kubectl run dns-test --image=busybox:1.28 --rm -it --restart=Never -- nslookup kubernetes.default
# Test external DNS kubectl run dns-test --image=busybox:1.28 --rm -it --restart=Never -- nslookup google.com
# Test service resolution kubectl run dns-test --image=busybox:1.28 --rm -it --restart=Never -- nslookup my-service.default.svc.cluster.local ```
All should resolve successfully.
Check CoreDNS metrics:
kubectl port-forward -n kube-system svc/kube-dns 9153:9153 &
curl http://localhost:9153/metrics | grep corednsQuick Diagnostic Script
```bash #!/bin/bash
echo "=== CoreDNS Pods ===" kubectl get pods -n kube-system -l k8s-app=kube-dns -o wide
echo -e "\n=== CoreDNS Service ===" kubectl get svc -n kube-system kube-dns
echo -e "\n=== CoreDNS Endpoints ===" kubectl get endpoints -n kube-system kube-dns
echo -e "\n=== CoreDNS Logs ===" kubectl logs -n kube-system -l k8s-app=kube-dns --tail=20
echo -e "\n=== DNS Test ===" kubectl run dns-test --image=busybox:1.28 --rm -it --restart=Never -- nslookup kubernetes.default 2>&1
echo -e "\n=== Pod DNS Config ===" kubectl run dns-test --image=busybox:1.28 --rm -it --restart=Never -- cat /etc/resolv.conf 2>&1
echo -e "\n=== CoreDNS ConfigMap ===" kubectl get configmap coredns -n kube-system -o yaml ```
Key Takeaways
- 1.CoreDNS pods must be running and ready in kube-system namespace
- 2.kube-dns service must have endpoints pointing to CoreDNS pods
- 3.Pod resolv.conf must point to kube-dns ClusterIP (10.96.0.10)
- 4.Network policies must allow DNS traffic to kube-system
- 5.Upstream DNS forwarding must be configured correctly
- 6.Resource limits must accommodate query volume
- 7.DNS loops occur when node resolv.conf points to cluster DNS
DNS issues cascade to all cluster workloads. Check CoreDNS pod health first, then service endpoints, then pod configuration. Most DNS problems stem from CoreDNS pods not running, network policies blocking traffic, or misconfigured upstream forwarding.
Related Articles
- [Fix Envoy Rate Limit Configuration with envoyproxy/ratelimit](envoyproxy-ratelimit-configuration-guide)
- [Fix Fix Argocd App Not Syncing Issue in Kubernetes](fix-argocd-app-not-syncing)
- [Fix Fix Argocd Sync Conflict Issue in Kubernetes](fix-argocd-sync-conflict)
- [Fix ArgoCD Sync Timeout](fix-argocd-sync-timeout)
- [How to Fix Cilium Identity Exhaustion and Endpoint Allocation Failed](fix-cilium-identity-exhaustion)
<script type="application/ld+json"> { "@context": "https://schema.org", "@type": "TechArticle", "headline": "Fix Kubernetes CoreDNS Not Resolving", "description": "Learn how to fix CoreDNS resolution failures in Kubernetes with solutions for pod issues, configuration errors, and network connectivity problems.", "url": "https://www.fixwikihub.com/fix-kubernetes-coredns-not-resolving", "publisher": { "@type": "Organization", "name": "FixWikiHub", "url": "https://www.fixwikihub.com" }, "author": { "@type": "Person", "name": "FixWikiHub Editorial Team" }, "datePublished": "2025-11-23T23:41:12.083Z", "dateModified": "2025-11-23T23:41:12.083Z" } </script>