Introduction
Nginx is running out of available file descriptors. Each connection, log file, and static file served requires a file descriptor. When Nginx hits the system's open file limit, it cannot accept new connections or serve files, resulting in errors and failed requests.
Symptoms
```bash $ tail -f /var/log/nginx/error.log 2024/01/01 12:00:00 [alert] 1234#1234: *67890 socket() failed (24: Too many open files) while connecting to upstream 2024/01/01 12:00:00 [crit] 1234#1234: *67891 open() "/var/www/html/index.html" failed (24: Too many open files)
$ curl http://localhost <html> <head><title>500 Internal Server Error</title></head> <body> <center><h1>500 Internal Server Error</h1></center> </body> </html>
$ dmesg | grep "open files" [12345.678901] nginx[1234]: segfault at 0 ip 0000000000000000 sp 00007ffc12345678 error 14
$ cat /proc/$(cat /var/run/nginx.pid)/limits | grep "open files" Max open files 1024 1024 files ```
Common Causes
- 1.Default limit too low: System default is often 1024 file descriptors
- 2.High traffic: Many concurrent connections exhaust descriptors
- 3.Keepalive connections: Long-lived connections holding descriptors
- 4.Many log files: Each log file uses a descriptor
- 5.Cache files: Proxy cache using many file descriptors
- 6.Worker processes: Each worker needs its own descriptors
- 7.No worker_rlimit_nofile: Nginx limit not configured
- 8.System limits not tuned: OS limits still at default
Step-by-Step Fix
- 1.Check logs for specific error messages
- 2.Verify configuration settings
- 3.Test network connectivity
- 4.Review recent changes
- 5.Apply corrective action
- 6.Verify the fix
Step 1: Check Current Limits
Identify the current limits:
```bash # Check Nginx process limits cat /proc/$(cat /var/run/nginx.pid)/limits | grep "open files"
# Check current file descriptors in use ls /proc/$(cat /var/run/nginx.pid)/fd | wc -l
# Check per-worker limits for pid in $(pgrep nginx); do echo "Worker $pid: $(ls /proc/$pid/fd 2>/dev/null | wc -l) files" done
# Check system-wide limit cat /proc/sys/fs/file-max
# Check soft and hard limits ulimit -Sn # Soft limit ulimit -Hn # Hard limit
# Check all limits for nginx user sudo -u nginx bash -c 'ulimit -a' ```
Step 2: Configure worker_rlimit_nofile
Set Nginx's file descriptor limit:
```nginx # In nginx.conf main context worker_processes auto; worker_rlimit_nofile 65535; # Set file descriptor limit for workers
events { worker_connections 4096; # Each connection uses a descriptor # Total possible connections = worker_processes * worker_connections # Should be <= worker_rlimit_nofile } ```
Calculate appropriate values: ```bash # Formula for worker_rlimit_nofile: # worker_rlimit_nofile >= worker_connections * 2 # (factor of 2 for connections + file serving)
# Example: # worker_connections = 4096 # worker_rlimit_nofile = 8192 (minimum) or 65535 (recommended)
# For high traffic: worker_rlimit_nofile 100000; worker_connections 50000; ```
Step 3: Increase System File Limits
Configure OS-level limits:
Option A: Using systemd (modern Linux)
``bash
# Edit nginx service configuration
sudo systemctl edit nginx
Add:
``ini
[Service]
LimitNOFILE=65535
LimitNOFILESoft=65535
```bash # Reload systemd sudo systemctl daemon-reload
# Restart nginx sudo systemctl restart nginx
# Verify cat /proc/$(cat /var/run/nginx.pid)/limits | grep "open files" ```
Option B: Using limits.conf
``bash
# Edit security limits
sudo nano /etc/security/limits.conf
Add:
``
nginx soft nofile 65535
nginx hard nofile 65535
root soft nofile 65535
root hard nofile 65535
* soft nofile 65535
* hard nofile 65535
Option C: Using sysctl
``bash
# Edit sysctl configuration
sudo nano /etc/sysctl.conf
Add:
``ini
fs.file-max = 1000000
fs.nr_open = 1000000
```bash # Apply changes sudo sysctl -p
# Verify cat /proc/sys/fs/file-max ```
Step 4: Configure PAM Limits
Ensure PAM applies limits:
```bash # Check if pam_limits is enabled grep pam_limits /etc/pam.d/*
# If not, add to common session sudo nano /etc/pam.d/common-session ```
Add:
``
session required pam_limits.so
# For SSH sessions
sudo nano /etc/pam.d/sshdAdd:
``
session required pam_limits.so
Step 5: Optimize Connection Settings
Reduce file descriptor usage:
```nginx # Optimize keepalive to free descriptors faster http { keepalive_timeout 30s; # Reduce from default 75s keepalive_requests 100; # Close after 100 requests
# Don't keepalive if limit approaching # Connection recycling reset_timedout_connection on;
# Optimize worker connections events { worker_connections 4096; # Adjust based on traffic use epoll; # Linux: efficient event model multi_accept on; # Accept multiple connections at once }
# Logging - can use descriptors access_log off; # Disable if not needed # Or use buffer access_log /var/log/nginx/access.log buffer=32k flush=5s; } ```
Step 6: Manage Cache File Descriptors
Control cache-related descriptors:
```nginx # Proxy cache uses file descriptors http { # Limit cache loader proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=cache:100m loader_files=1000; # Files per loader iteration proxy_cache_max_range_offset 1m; # Don't cache large ranges
# Open file cache (for static files) open_file_cache max=10000 inactive=30s; open_file_cache_valid 60s; open_file_cache_min_uses 2; open_file_cache_errors on;
# Disable access log to save descriptors # access_log off; } ```
Step 7: Monitor File Descriptor Usage
Track descriptor usage:
```bash # Real-time monitoring watch -n 1 'ls /proc/$(cat /var/run/nginx.pid)/fd | wc -l'
# Per-process monitoring watch -n 1 'for pid in $(pgrep nginx); do echo -n "Worker $pid: "; ls /proc/$pid/fd 2>/dev/null | wc -l; done'
# Check open files by nginx sudo lsof -p $(cat /var/run/nginx.pid) | wc -l
# Check what files are open sudo lsof -p $(cat /var/run/nginx.pid) | head -20
# Check types of open files sudo lsof -p $(cat /var/run/nginx.pid) | awk '{print $5}' | sort | uniq -c ```
Step 8: Check for File Descriptor Leaks
Identify if Nginx is leaking:
```bash # Monitor descriptor growth ls /proc/$(cat /var/run/nginx.pid)/fd | wc -l sleep 60 ls /proc/$(cat /var/run/nginx.pid)/fd | wc -l # If growing continuously, there's a leak
# Check what's using descriptors sudo lsof -p $(cat /var/run/nginx.pid) | grep -E "REG|sock"
# Check for zombie connections ss -tnp | grep nginx | grep -c ESTAB
# Check worker status sudo nginx -t 2>&1 ps aux | grep nginx ```
Step 9: Emergency Recovery
Recover when hitting limits:
```bash # Immediate: restart nginx to clear state sudo systemctl restart nginx
# Check error log for cause tail -100 /var/log/nginx/error.log | grep "open files"
# Temporary: kill excess connections sudo ss -K dst 192.168.1.0/24 dport = 80
# Reduce connections # Edit nginx.conf temporarily sudo nano /etc/nginx/nginx.conf ```
events {
worker_connections 1024; # Reduce temporarily
}sudo systemctl reload nginxStep 10: Configure for High Traffic
Production-ready configuration:
```nginx # High traffic configuration worker_processes auto; worker_rlimit_nofile 100000;
events { worker_connections 50000; use epoll; multi_accept on; }
http { # Optimize for high connections keepalive_timeout 30s; keepalive_requests 100; keepalive_disable msie6; reset_timedout_connection on;
# Open file cache open_file_cache max=20000 inactive=20s; open_file_cache_valid 30s; open_file_cache_min_uses 2; open_file_cache_errors on;
# Buffer access log access_log /var/log/nginx/access.log buffer=64k flush=1m;
# Sendfile reduces descriptor usage sendfile on; tcp_nopush on; tcp_nodelay on; } ```
System configuration:
``bash
# /etc/sysctl.d/99-nginx.conf
fs.file-max = 1000000
fs.nr_open = 1000000
net.core.somaxconn = 65535
net.ipv4.tcp_max_syn_backlog = 65535
net.ipv4.ip_local_port_range = 1024 65535
Verification
Confirm limits are increased:
```bash # Check Nginx limits cat /proc/$(cat /var/run/nginx.pid)/limits | grep "open files" # Should show: Max open files 65535 65535 files
# Check current usage ls /proc/$(cat /var/run/nginx.pid)/fd | wc -l
# Check system limits ulimit -n # Should show: 65535
# Verify configuration sudo nginx -t
# Test under load ab -n 10000 -c 100 http://localhost/
# Monitor during load watch -n 1 'ls /proc/$(cat /var/run/nginx.pid)/fd | wc -l'
# Check no errors in log tail -f /var/log/nginx/error.log | grep -i "open files" ```
File descriptor limits should now be sufficient for your traffic load.
Related Articles
- [Nginx troubleshooting: Fix Lambda Permission Denied - Complete ](fix-lambda-permission-denied)
- [Nginx web server troubleshooting: Fix Client Max Body Size Large Upload Nginx Issue ](client-max-body-size-large-upload-nginx)
- [Fix Apache 502 Proxy Error](fix-apache-502-proxy-error)
- [Fix Apache LogLevel Core Debug Configuration](fix-apache-loglevel-core-debug)
- [Fix Cloudflare 502 Bad Gateway Error](fix-cloudflare-502-bad-gateway)
<script type="application/ld+json"> { "@context": "https://schema.org", "@type": "TechArticle", "headline": "Nginx Too Many Open Files", "description": "Resolve Nginx too many open files errors with worker_rlimit_nofile, systemd limits, and kernel parameter tuning", "url": "https://www.fixwikihub.com/fix-nginx-open-files-limit-exceeded", "publisher": { "@type": "Organization", "name": "FixWikiHub", "url": "https://www.fixwikihub.com" }, "author": { "@type": "Person", "name": "FixWikiHub Editorial Team" }, "datePublished": "2025-11-19T13:18:39.564Z", "dateModified": "2025-11-19T13:18:39.564Z" } </script>