Home / AWS / Fix AWS EKS IAM Role for Service Account (IRSA) Not Working

AWS

Fix AWS EKS IAM Role for Service Account (IRSA) Not Working

Resolve EKS IRSA issues by configuring OIDC provider, IAM roles, and service account annotations correctly.

Published: Apr 1, 20267 min readBy FixWikiHub Editorial Team

Abstract illustration for a troubleshooting knowledge base category.

Introduction

IAM Roles for Service Accounts (IRSA) lets Kubernetes pods assume AWS IAM roles. When IRSA fails, pods can't access AWS services like S3, DynamoDB, or Secrets Manager despite having the correct service account annotation.

Symptoms

Pod can't access AWS services:

```bash $ kubectl logs my-pod

botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the GetObject operation ```

AWS SDK not finding credentials:

```bash $ kubectl exec -it my-pod -- env | grep AWS

AWS_ROLE_ARN=arn:aws:iam::123456789:role/my-role AWS_WEB_IDENTITY_TOKEN_FILE=/var/run/secrets/eks.amazonaws.com/serviceaccount/token # Token file exists but AWS still returns AccessDenied ```

Service account annotation missing:

```bash $ kubectl get sa my-service-account -o yaml

# No eks.amazonaws.com/role-arn annotation ```

Common Causes

1.OIDC provider not configured - Cluster doesn't have OIDC identity provider
2.Wrong trust policy - Role doesn't trust the correct OIDC provider
3.Service account not annotated - Missing role ARN annotation
4.Incorrect subject in trust policy - Wrong service account name or namespace
5.AWS SDK version issue - Old SDK doesn't support web identity
6.Token file missing - Pod doesn't have projected service account token

Step-by-Step Fix

1.Check logs for specific error messages
2.Verify configuration settings
3.Test network connectivity
4.Review recent changes
5.Apply corrective action
6.Verify the fix

Step 1: Check OIDC Provider Exists

```bash # Get cluster OIDC issuer URL aws eks describe-cluster --name my-cluster \ --query 'cluster.identity.oidc.issuer'

# Example: https://oidc.eks.region.amazonaws.com/id/EXAMPLED539D18

# Check if IAM OIDC provider exists aws iam list-open-id-connect-providers \ --query 'OpenIDConnectProviderList[*].Arn'

# If missing, create it eksctl utils associate-iam-oidc-provider --cluster my-cluster --approve

# Or manually: OIDC_ID=$(aws eks describe-cluster --name my-cluster \ --query 'cluster.identity.oidc.issuer' --output text | cut -d'/' -f5)

aws iam create-open-id-connect-provider \ --url https://oidc.eks.region.amazonaws.com/id/$OIDC_ID \ --client-id-list sts.amazonaws.com \ --thumbprint-list 9e99a48a9960b14926bb7f3b02e22da2b0ab7280 ```

Step 2: Verify Service Account Annotation

```bash # Check service account annotation kubectl get sa my-service-account -n my-namespace -o yaml

# Should have annotation: metadata: annotations: eks.amazonaws.com/role-arn: arn:aws:iam::ACCOUNT:role/my-role

# Add annotation if missing kubectl annotate sa my-service-account -n my-namespace \ eks.amazonaws.com/role-arn=arn:aws:iam::ACCOUNT:role/my-role

# Restart pods to pick up new annotation kubectl rollout restart deployment/my-deployment -n my-namespace ```

Step 3: Check IAM Role Trust Policy

```bash # Get role trust policy aws iam get-role --role-name my-role \ --query 'Role.AssumeRolePolicyDocument'

# Must trust the OIDC provider with correct subject: { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Federated": "arn:aws:iam::ACCOUNT:oidc-provider/oidc.eks.region.amazonaws.com/id/OIDC_ID" }, "Action": "sts:AssumeRoleWithWebIdentity", "Condition": { "StringEquals": { "oidc.eks.region.amazonaws.com/id/OIDC_ID:sub": "system:serviceaccount:my-namespace:my-service-account" } } } ] } ```

Step 4: Fix Trust Policy Subject

The subject must match exactly: system:serviceaccount:NAMESPACE:SERVICE_ACCOUNT_NAME

```bash # Create role with correct trust policy cat > trust-policy.json << EOF { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Federated": "arn:aws:iam::ACCOUNT:oidc-provider/oidc.eks.region.amazonaws.com/id/OIDC_ID" }, "Action": "sts:AssumeRoleWithWebIdentity", "Condition": { "StringEquals": { "oidc.eks.region.amazonaws.com/id/OIDC_ID:sub": "system:serviceaccount:my-namespace:my-service-account" } } } ] } EOF

aws iam update-assume-role-policy \ --role-name my-role \ --policy-document file://trust-policy.json ```

Step 5: Verify Pod Has Token File

```bash # Check if pod has token mounted kubectl exec -it my-pod -n my-namespace -- ls -la /var/run/secrets/eks.amazonaws.com/serviceaccount/

# Should show: # token # aws_iam_role

# Check token content kubectl exec -it my-pod -n my-namespace -- cat /var/run/secrets/eks.amazonaws.com/serviceaccount/token

# Decode JWT to verify claims kubectl exec -it my-pod -n my-namespace -- cat /var/run/secrets/eks.amazonaws.com/serviceaccount/token | cut -d'.' -f2 | base64 -d | jq ```

Step 6: Check IAM Role Permissions

```bash # Verify role has the needed permissions aws iam list-attached-role-policies --role-name my-role

# Test if role can be assumed aws sts assume-role-with-web-identity \ --role-arn arn:aws:iam::ACCOUNT:role/my-role \ --role-session-name test-session \ --web-identity-token TOKEN_VALUE ```

Add permissions if missing:

bash

aws iam attach-role-policy \
  --role-name my-role \
  --policy-arn arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess

Step 7: Debug with AWS CLI in Pod

```bash # Run test pod with AWS CLI kubectl run aws-cli --image=amazon/aws-cli --rm -it --restart=Never \ --serviceaccount=my-service-account \ -- aws s3 ls

# Should list S3 buckets if IRSA working

# Check environment variables kubectl exec -it my-pod -- env | grep AWS

# Should show: # AWS_ROLE_ARN=arn:aws:iam::... # AWS_WEB_IDENTITY_TOKEN_FILE=/var/run/seeks/eks.amazonaws.com/serviceaccount/token ```

Step 8: Check for Multiple Service Accounts

```bash # If pod uses multiple service accounts, only first one is used kubectl get pod my-pod -o jsonpath='{.spec.serviceAccountName}'

# Ensure the annotated service account is the one used by the pod ```

Step 9: Verify AWS SDK Version

```bash # AWS SDK must support web identity # Python boto3 >= 1.9.0 # Java SDK >= 1.11.704 # Node.js SDK >= 2.425.0 # Go SDK >= 1.23.0

# Check SDK version in application logs or dependencies # For Python: kubectl exec -it my-pod -- python -c "import boto3; print(boto3.__version__)"

# For Node.js: kubectl exec -it my-pod -- npm list aws-sdk ```

Step 10: Test with Minimal Example

```bash # Create test deployment kubectl apply -f - <<EOF apiVersion: v1 kind: ServiceAccount metadata: name: test-sa annotations: eks.amazonaws.com/role-arn: arn:aws:iam::ACCOUNT:role/test-role --- apiVersion: apps/v1 kind: Deployment metadata: name: test-irsa spec: selector: matchLabels: app: test-irsa template: metadata: labels: app: test-irsa spec: serviceAccountName: test-sa containers: - name: aws-cli image: amazon/aws-cli command: ["sleep", "3600"] EOF

# Test S3 access kubectl exec -it deployment/test-irsa -- aws s3 ls ```

IRSA Troubleshooting Checklist

[ ] OIDC provider exists in IAM
[ ] Service account has role-arn annotation
[ ] Role trust policy references correct OIDC provider
[ ] Subject matches system:serviceaccount:NAMESPACE:SA_NAME
[ ] Pod uses correct service account
[ ] AWS SDK version supports web identity
[ ] Role has necessary AWS permissions

Verification

```bash # Test AWS service access from pod kubectl exec -it my-pod -n my-namespace -- aws s3 ls

# Should list buckets without AccessDenied

# Check CloudTrail for AssumeRoleWithWebIdentity aws cloudtrail lookup-events \ --lookup-attributes AttributeKey=EventName,AttributeValue=AssumeRoleWithWebIdentity \ --max-items 5 ```

[Fix AWS EKS Node Not Joining](/articles/fix-aws-eks-node-not-joining)
[Fix AWS EKS Pod Networking Failed](/articles/fix-aws-eks-pod-networking-failed)
[Fix AWS S3 Bucket Policy Denied](/articles/fix-aws-s3-bucket-policy-denied)

[AWS troubleshooting: Fix IAM Permission Denied - Complete Tro](fix-iam-permission-denied)
[AWS cloud troubleshooting: AWS ACM Certificate Pending Validation Because the](aws-acm-certificate-pending-validation-wrong-route53-zone)
[AWS cloud troubleshooting: AWS ALB Returns 502 Because the Target Closed the ](aws-alb-502-target-closed-connection-keepalive-timeout-mismatch)
[AWS cloud troubleshooting: Fix AWS ALB CreateListener TargetGroupNotFound Err](aws-alb-createlistener-targetgroupnotfound)
[AWS cloud troubleshooting: Fix Aws Alb Lambda 502 Bad Gateway Issue in AWS](aws-alb-lambda-502-bad-gateway)

Was this guide helpful?

Related search paths

People also search for

If the symptom is close but not identical, these search paths usually surface the right neighboring fixes faster than scrolling the full archive.

AWS EKS IAM Role for Service Account (IRSA) Not Working AWS EKS IAM Role for Service Account (IRSA) Not Working AWS AWS EKS IAM Role for Service Account (IRSA) Not Working troubleshooting AWS EKS IAM Role for Service Account (IRSA) Not Working fix Resolve EKS IRSA issues by configuring OIDC provider, IAM roles, and service account annotations correctly AWS Resolve EKS IRSA issues by configuring OIDC provider, IAM roles, and service account annotations correctly

Explore Related Topics

Browse Guides from Other Categories

Discover troubleshooting guides from related categories to expand your knowledge.

FAQ

AWS Troubleshooting FAQs

Common questions about troubleshooting and preventing similar issues

How do I know if this aws-errors troubleshooting guide applies to my situation?

This guide is designed for aws-errors issues. If you're experiencing similar symptoms described in the article, follow the step-by-step instructions. Start with the most common causes and work through the diagnostic process.

Is it safe to follow these aws-errors troubleshooting steps?

Yes, all steps are designed to be safe and non-destructive. We recommend creating backups before making significant changes and testing each step before proceeding to the next.

How long does it typically take to resolve this type of aws-errors issue?

Most aws-errors issues can be resolved within 30 minutes to 2 hours, depending on the complexity and root cause. Follow the troubleshooting flow to identify and fix the problem efficiently.

How can I prevent this aws-errors issue from happening again?

Regular maintenance, monitoring, and following best practices for aws-errors configuration can help prevent recurrence. Consider implementing automated checks and alerts for early detection.

Written by

FixWikiHub Editorial Team

Our editorial team consists of experienced DevOps engineers, systems administrators, and cloud architects with hands-on experience in production environments across AWS, Azure, GCP, and on-premises infrastructure.

Every guide undergoes technical review for accuracy and is updated when software versions, commands, or best practices change.

Last updated: Apr 1, 2026

About our team

Important Notice

Disclaimer & Safety Guidelines

The troubleshooting steps in this guide are provided for educational and informational purposes. Before applying any changes to production systems:

Test in a staging environment first — Always verify commands and configurations in a non-production environment before deploying to live systems.
Create backups — Ensure you have current backups of databases, configurations, and critical files before making changes.
Understand the impact — Review how each step may affect your specific environment, dependencies, and users.
Consult official documentation — This guide supplements, but does not replace, official vendor documentation and best practices.

FixWikiHub is not responsible for any damages arising from the use of this content. See our Terms of Use for more information.

Resources

Official Documentation & Further Reading

For authoritative information, consult the official documentation for the technologies discussed in this guide. Our troubleshooting content supplements, but does not replace, vendor documentation.

AWS Documentation — Official Amazon Web Services guides and API references
Kubernetes Documentation — Official Kubernetes documentation
Nginx Documentation — Official Nginx web server documentation
Apache Documentation — Official Apache HTTP Server documentation
Docker Documentation — Official Docker container documentation

Fix AWS EKS IAM Role for Service Account (IRSA) Not Working

Introduction

Symptoms

Common Causes

Step-by-Step Fix

Step 1: Check OIDC Provider Exists

Step 2: Verify Service Account Annotation

Step 3: Check IAM Role Trust Policy

Step 4: Fix Trust Policy Subject

Step 5: Verify Pod Has Token File

Step 6: Check IAM Role Permissions

Step 7: Debug with AWS CLI in Pod

Step 8: Check for Multiple Service Accounts

Step 9: Verify AWS SDK Version

Step 10: Test with Minimal Example

IRSA Troubleshooting Checklist

Verification

People also search for

Browse Guides from Other Categories

WordPress

SSL

DNS

AWS Troubleshooting FAQs

FixWikiHub Editorial Team

Disclaimer & Safety Guidelines

Official Documentation & Further Reading

Fix AWS EKS IAM Role for Service Account (IRSA) Not Working

Introduction

Symptoms

Common Causes

Step-by-Step Fix

Step 1: Check OIDC Provider Exists

Step 2: Verify Service Account Annotation

Step 3: Check IAM Role Trust Policy

Step 4: Fix Trust Policy Subject

Step 5: Verify Pod Has Token File

Step 6: Check IAM Role Permissions

Step 7: Debug with AWS CLI in Pod

Step 8: Check for Multiple Service Accounts

Step 9: Verify AWS SDK Version

Step 10: Test with Minimal Example

IRSA Troubleshooting Checklist

Verification

Related Issues

Related Articles

People also search for

Share this guide

More AWS Troubleshooting Guides

Browse Guides from Other Categories

AWS Troubleshooting FAQs

FixWikiHub Editorial Team

Disclaimer & Safety Guidelines

Official Documentation & Further Reading