Introduction

EKS uses the Amazon VPC CNI plugin to assign IP addresses to pods from the VPC subnet. When pod networking fails, pods can't communicate with each other, services, or external resources. This manifests as connection timeouts, DNS failures, or pods stuck in ContainerCreating.

Symptoms

Pod stuck in ContainerCreating:

```bash $ kubectl get pods

NAME READY STATUS RESTARTS AGE my-app-12345-abcde 0/1 ContainerCreating 0 5m

$ kubectl describe pod my-app-12345-abcde

Warning FailedCreatePodSandBox Pod sandbox changed, it will be killed and re-created. Error: failed to set up sandbox container: ADD failed: failed to assign an IP address to container ```

Inter-pod connectivity failure:

```bash $ kubectl exec -it pod-a -- curl pod-b-service

curl: (7) Failed to connect to pod-b-service port 80: Connection timed out ```

DNS resolution failure:

```bash $ kubectl exec -it pod-a -- nslookup kubernetes

;; connection timed out; no servers could be reached ```

CNI plugin errors:

```bash $ kubectl logs -n kube-system -l k8s-app=aws-node

E0115 10:00:00.000000 ipamd.go:123] Failed to get ENI IP pool: InsufficientCidrBlocks ```

Common Causes

  1. 1.IP address exhaustion - Subnet ran out of available IPs
  2. 2.VPC CNI not running - aws-node DaemonSet issue
  3. 3.Security group restrictions - Blocking pod-to-pod traffic
  4. 4.Network policies blocking - Kubernetes NetworkPolicy denying traffic
  5. 5.ENI attachment limit - Instance can't attach more ENIs
  6. 6.Wrong CNI configuration - Misconfigured IPAMD settings
  7. 7.CoreDNS issues - DNS not resolving service names

Step-by-Step Fix

  1. 1.Check logs for specific error messages
  2. 2.Verify configuration settings
  3. 3.Test network connectivity
  4. 4.Review recent changes
  5. 5.Apply corrective action
  6. 6.Verify the fix

Step 1: Check VPC CNI DaemonSet

```bash # Check aws-node pods kubectl get pods -n kube-system -l k8s-app=aws-node

# Should show one per node with Running status NAME READY STATUS RESTARTS AGE aws-node-abc12 1/1 Running 0 1d

# Check CNI logs kubectl logs -n kube-system -l k8s-app=aws-node --tail=100

# Restart CNI pods if needed kubectl rollout restart daemonset/aws-node -n kube-system ```

Step 2: Check Subnet IP Availability

```bash # Get subnet IDs used by EKS aws eks describe-cluster --name my-cluster \ --query 'cluster.resourcesVpcConfig.subnetIds'

# Check available IPs in subnets aws ec2 describe-subnets --subnet-ids subnet-12345 \ --query 'Subnets[*].[SubnetId,AvailableIpAddressCount,CidrBlock]'

# If AvailableIpAddressCount is low (< 10), need more IPs: # 1. Create new subnet with larger CIDR # 2. Or add secondary CIDR to VPC ```

Add secondary CIDR:

```bash # Associate additional CIDR to VPC aws ec2 associate-vpc-cidr-block \ --vpc-id vpc-12345 \ --cidr-block 100.64.0.0/16

# Create new subnet in secondary CIDR aws ec2 create-subnet \ --vpc-id vpc-12345 \ --cidr-block 100.64.0.0/18 \ --availability-zone us-east-1a

# Tag for EKS aws ec2 create-tags --resources subnet-new \ --tags Key=kubernetes.io/cluster/my-cluster,Value=shared ```

Step 3: Check ENI Attachment Limits

```bash # Each EC2 instance has ENI limits # Get instance type limits aws ec2 describe-instance-types --instance-types m5.large \ --query 'InstanceTypes[*].[InstanceType,NetworkInfo.MaximumNetworkInterfaces,NetworkInfo.Ipv4AddressesPerInterface]'

# Max ENIs × IPs per ENI = max pods per node

# Check current ENI usage on node INSTANCE_ID=$(kubectl get node NODE_NAME -o jsonpath='{.spec.providerID}' | cut -d'/' -f5) aws ec2 describe-instances --instance-ids $INSTANCE_ID \ --query 'Reservations[*].Instances[*].NetworkInterfaces[*].[NetworkInterfaceId,PrivateIpAddress]' ```

Step 4: Check Node Security Groups

```bash # Get node security groups aws ec2 describe-instances --instance-ids $INSTANCE_ID \ --query 'Reservations[*].Instances[*].SecurityGroups[*].GroupId'

# Pods use node's primary security group (by default) # Check rules allow pod traffic aws ec2 describe-security-groups --group-ids sg-12345 \ --query 'SecurityGroups[*].IpPermissions'

# Need to allow: # - Intra-VPC traffic (for pod-to-pod) # - Node-to-node traffic # - Pod port ranges ```

Add security group rules:

```bash # Allow all traffic from VPC CIDR (pod-to-pod) aws ec2 authorize-security-group-ingress \ --group-id sg-12345 \ --protocol -1 \ --cidr 10.0.0.0/16

# Or use security group self-reference aws ec2 authorize-security-group-ingress \ --group-id sg-12345 \ --protocol -1 \ --source-group sg-12345 ```

Step 5: Check Network Policies

```bash # List network policies kubectl get networkpolicies --all-namespaces

# Check specific policy kubectl get networkpolicy default-deny -n my-namespace -o yaml

# Network policies can block: # - Pod-to-pod traffic # - DNS resolution # - External access

# Temporarily delete restrictive policy kubectl delete networkpolicy restrict-policy -n my-namespace ```

Step 6: Test Pod-to-Pod Connectivity

```bash # Get pod IPs kubectl get pods -o wide

# Test connectivity between pods kubectl exec -it pod-a -- ping POD_B_IP

# Test service resolution kubectl exec -it pod-a -- curl http://pod-b-service:80

# Test DNS kubectl exec -it pod-a -- nslookup kubernetes.default

# If DNS fails, check CoreDNS kubectl get pods -n kube-system -l k8s-app=kube-dns kubectl logs -n kube-system -l k8s-app=kube-dns ```

Step 7: Check CNI Configuration

```bash # Get CNI configuration kubectl get ds aws-node -n kube-system -o yaml | grep -A 30 env

# Key environment variables: # - AWS_VPC_K8S_CNI_LOGLEVEL: Logging level # - AWS_VPC_K8S_CNI_CUSTOM_NETWORK_CFG: Custom networking # - ENI_CONFIG_LABEL_DEF: ENI config label # - WARM_ENI_TARGET: Pre-allocated ENIs

# Check IPAMD stats kubectl exec -n kube-system aws-node-abc12 -- curl -s localhost:61679/v1/enis | jq ```

Step 8: Enable Custom Networking (for IP exhaustion)

```bash # Create ENIConfig for custom networking kubectl apply -f - <<EOF apiVersion: crd.k8s.amazonaws.com/v1alpha1 kind: ENIConfig metadata: name: us-east-1a spec: securityGroups: - sg-12345 subnet: subnet-secondary EOF

# Annotate nodes kubectl annotate node NODE_NAME k8s.amazonaws.com/eniConfig=us-east-1a

# Enable custom networking kubectl set env daemonset/aws-node -n kube-system AWS_VPC_K8S_CNI_CUSTOM_NETWORK_CFG=true ```

Step 9: Check CoreDNS

```bash # Check CoreDNS pods kubectl get pods -n kube-system -l k8s-app=kube-dns

# Check CoreDNS logs kubectl logs -n kube-system -l k8s-app=kube-dns

# Test DNS from pod kubectl run dnstest --image=busybox --rm -it --restart=Never -- nslookup kubernetes.default

# If failing, check CoreDNS ConfigMap kubectl get configmap coredns -n kube-system -o yaml ```

Step 10: Debug with Network Tools Pod

```bash # Deploy network debug pod kubectl apply -f - <<EOF apiVersion: v1 kind: Pod metadata: name: netshoot spec: containers: - name: netshoot image: nicolaka/netshoot command: ["sleep", "3600"] hostNetwork: false EOF

# Test from debug pod kubectl exec -it netshoot -- bash

# Inside pod: ip addr show ip route curl -v http://kubernetes.default.svc nslookup kubernetes.default curl http://POD_IP:PORT ```

Common EKS Networking Errors

ErrorCauseFix
InsufficientCidrBlocksSubnet out of IPsAdd secondary CIDR
ENI attachment failedInstance limitUse larger instance
Connection timeoutSecurity groupAdd SG rules
DNS failureCoreDNS issueCheck CoreDNS pods
Network unreachableNetworkPolicyReview/delete policy

Verification

```bash # Check pods can get IPs kubectl get pods -o wide

# All pods should have valid IPs from subnet range

# Test pod-to-pod communication kubectl exec -it pod-a -- curl http://pod-b-service

# Test DNS resolution kubectl exec -it pod-a -- nslookup kubernetes.default

# Check CNI is healthy kubectl logs -n kube-system -l k8s-app=aws-node --tail=20

# Should not show errors ```

  • [Fix AWS EKS Node Not Joining](/articles/fix-aws-eks-node-not-joining)
  • [Fix AWS EKS Cluster Autoscaler Not Scaling](/articles/fix-aws-eks-cluster-autoscaler-not-scaling)
  • [Fix Kubernetes Pod Stuck in Pending](/articles/fix-kubernetes-pod-crashloopbackoff)
  • [AWS troubleshooting: Fix IAM Permission Denied - Complete Tro](fix-iam-permission-denied)
  • [AWS cloud troubleshooting: AWS ACM Certificate Pending Validation Because the](aws-acm-certificate-pending-validation-wrong-route53-zone)
  • [AWS cloud troubleshooting: AWS ALB Returns 502 Because the Target Closed the ](aws-alb-502-target-closed-connection-keepalive-timeout-mismatch)
  • [AWS cloud troubleshooting: Fix AWS ALB CreateListener TargetGroupNotFound Err](aws-alb-createlistener-targetgroupnotfound)
  • [AWS cloud troubleshooting: Fix Aws Alb Lambda 502 Bad Gateway Issue in AWS](aws-alb-lambda-502-bad-gateway)

<script type="application/ld+json"> { "@context": "https://schema.org", "@type": "TechArticle", "headline": "Fix AWS EKS Pod Networking Failed", "description": "Troubleshoot EKS pod networking failures. Fix VPC CNI issues, IP exhaustion, security groups, and network policies.", "url": "https://www.fixwikihub.com/fix-aws-eks-pod-networking-failed", "publisher": { "@type": "Organization", "name": "FixWikiHub", "url": "https://www.fixwikihub.com" }, "author": { "@type": "Person", "name": "FixWikiHub Editorial Team" }, "datePublished": "2026-04-01T23:50:29.411Z", "dateModified": "2026-04-01T23:50:29.411Z" } </script>