Introduction

When directories used by Ansible Tower or AWX are moved or renamed (such as relocating project files, inventory sources, or custom modules from one path to another), the runtime may continue attempting to access files from the old path. This results in "file not found" errors, use of stale data from remaining files at the old location, or confusing behavior where some operations succeed and others fail depending on whether the old path still has remnants.

This commonly occurs during: - Storage migration to new volumes - Reorganizing project directory structures - Moving from local storage to network-attached storage - Changing the projects_root or job_status_path configuration - Kubernetes volume mount path changes

Symptoms

Playbooks fail with file not found errors:

yaml
- name: Include shared tasks
  include_tasks: "{{ playbook_dir }}/../shared/tasks/main.yml"

Error:

bash
TASK [Include shared tasks] *****************************************************
fatal: [localhost]: FAILED! => {
    "msg": "Could not find file '/var/lib/awx/projects/123/shared/tasks/main.yml'"
}

But the file exists at the new location:

bash
$ ls -la /opt/tower/projects/123/shared/tasks/main.yml
-rw-r--r-- 1 awx awx 1024 Mar 15 14:00 /opt/tower/projects/123/shared/tasks/main.yml

Tower project sync shows path issues:

``` TASK [Update project from Git] ********** changed: [localhost]

TASK [Check playbook exists] ************ fatal: [localhost]: FAILED! => {"msg": "The playbook /var/lib/awx/projects/123/playbook.yml does not exist"} ```

In Tower logs:

bash
2024-03-15 14:23:45,123 ERROR    awx.main.tasks Project path /var/lib/awx/projects/123 not found
2024-03-15 14:24:12,456 WARNING  awx.main.models.project Using cached project from /var/lib/awx/projects/123 (path may be stale)
2024-03-15 14:25:01,789 ERROR    awx.main.dispatch FileNotFoundError: [Errno 2] No such file or directory: '/var/lib/awx/projects/123/ansible.cfg'

Custom modules not found:

bash
TASK [Run custom module] ********************************************************
fatal: [localhost]: FAILED! => {
    "msg": "The module custom_health_check was not found in configured module paths"
}
# Module exists at new path but not old path

Inventory source errors:

bash
$ tower-cli inventory_source update 42
Error: Cannot find inventory file at /var/lib/awx/projects/456/inventory.ini

Common Causes

Old paths persist due to several factors:

  1. 1.Tower configuration not updated: Settings like PROJECTS_ROOT, JOBOUTPUT_ROOT, or AWX_ISOLATION_SHOW_PATHS still reference the old directory structure.
  2. 2.Symlink broken during move: If directories were accessed via symlinks that weren't updated, the symlink may point to a non-existent location.
  3. 3.Database path references: Tower's database stores absolute paths for projects, inventories, and job artifacts. Moving directories without updating these records causes mismatches.
  4. 4.Environment variables in worker processes: Celery workers have cached environment variables pointing to old paths from when they started.
  5. 5.Ansible.cfg search paths: Ansible's configuration file search path includes the project directory, and if this is hardcoded, it references the old location.
  6. 6.Module search path not updated: Custom modules may be in the old path but Ansible's ANSIBLE_LIBRARY or module_path settings still reference it.
  7. 7.File descriptor caching: Some file handles or directory descriptors may be cached by Python or the OS.

Step-by-Step Fix

Step 1: Identify All Path References

Find all references to the old path:

```bash OLD_PATH="/var/lib/awx/projects" NEW_PATH="/opt/tower/projects"

# Check Tower settings grep -r "$OLD_PATH" /etc/tower/ grep -r "$OLD_PATH" /var/lib/awx/venv/awx/lib/python*/site-packages/awx/

# Check environment variables sudo cat /proc/$(pgrep -f "celery.*worker" | head -1)/environ | tr '\0' '\n' | grep -E "PROJECT|AWX|ANSIBLE"

# Check database path references sudo -u postgres psql -d awx -c " SELECT table_name, column_name FROM information_schema.columns WHERE column_name LIKE '%path%' OR column_name LIKE '%dir%';"

# Find actual path values in database sudo -u postgres psql -d awx -c " SELECT 'main_project' as tbl, id, local_path FROM main_project WHERE local_path LIKE '%$OLD_PATH%';"

sudo -u postgres psql -d awx -c " SELECT 'main_unifiedjob' as tbl, id, job_args FROM main_unifiedjob WHERE job_args::text LIKE '%$OLD_PATH%' LIMIT 10;"

# Check ansible.cfg files find /etc/tower -name "ansible.cfg" -exec grep -l "$OLD_PATH" {} \;

# Check for symlinks ls -la /var/lib/awx/projects 2>/dev/null ls -la /opt/tower/projects 2>/dev/null ```

Step 2: Update Tower Configuration

Modify all configuration files:

```bash # Backup current settings sudo cp /etc/tower/settings.py /etc/tower/settings.py.bak

# Update paths in settings sudo sed -i "s|$OLD_PATH|$NEW_PATH|g" /etc/tower/settings.py

# Verify changes grep -E "PROJECTS_ROOT|JOBOUTPUT_ROOT|AWX_ISOLATION" /etc/tower/settings.py

# Manual edit for specific settings sudo vim /etc/tower/settings.py

# Update these settings: PROJECTS_ROOT = '/opt/tower/projects' JOBOUTPUT_ROOT = '/opt/tower/job_output' AWX_ISOLATION_SHOW_PATHS = ['/opt/tower/projects']

# Update supervisord configuration sudo vim /etc/supervisord.d/tower.ini # Check for any hardcoded paths

# Update systemd service files if applicable sudo systemctl edit tower # Add environment overrides if needed [Service] Environment="PROJECTS_ROOT=/opt/tower/projects" ```

Step 3: Update Database Path References

Fix paths stored in the database:

```bash # Connect to database sudo -u postgres psql -d awx

# Start transaction BEGIN;

-- Update project paths UPDATE main_project SET local_path = REPLACE(local_path, '/var/lib/awx/projects/', '/opt/tower/projects/') WHERE local_path LIKE '/var/lib/awx/projects/%';

-- Check affected rows SELECT id, name, local_path FROM main_project WHERE local_path LIKE '/opt/tower/projects/%';

-- Update job event paths if applicable UPDATE main_jobevent SET job_args = REPLACE(job_args::text, '/var/lib/awx/projects/', '/opt/tower/projects/')::jsonb WHERE job_args::text LIKE '%/var/lib/awx/projects/%';

-- Update inventory source paths UPDATE main_inventorysource SET source_path = REPLACE(source_path, '/var/lib/awx/projects/', '/opt/tower/projects/') WHERE source_path LIKE '/var/lib/awx/projects/%';

-- Commit if correct COMMIT; -- Or rollback if issues -- ROLLBACK; ```

Step 4: Create Symlink or Remove Old Path

Either bridge the gap with symlinks or clean up:

```bash # Option 1: Create symlink from old to new path sudo mv /var/lib/awx/projects /var/lib/awx/projects.old 2>/dev/null sudo ln -s /opt/tower/projects /var/lib/awx/projects

# Verify symlink ls -la /var/lib/awx/projects # Should show: projects -> /opt/tower/projects

# Test access through both paths ls /var/lib/awx/projects/123/ ls /opt/tower/projects/123/

# Option 2: Remove old path completely (if no other references) sudo rm -rf /var/lib/awx/projects

# Then restart services to pick up new configuration sudo ansible-tower-service restart ```

Step 5: Clear Caches and Restart

Force all processes to use new paths:

```bash # Clear Redis cache redis-cli FLUSHDB

# Clear Django cache sudo -u awx awx-manage clear_cache

# Clear project cache sudo rm -rf /var/lib/awx/.cache/*

# Restart all Tower services sudo ansible-tower-service restart

# Or restart individual components sudo supervisorctl restart all

# Verify services started sudo supervisorctl status

# Check logs for path errors sudo tail -f /var/log/tower/task_system.log | grep -i "path|not found|no such file" ```

Step 6: Verify New Path Works

Test the new path is being used:

```bash # Check environment in new workers ps aux | grep "celery.*worker" | awk '{print $2}' | head -1 | \ xargs sudo cat /proc/{}/environ | tr '\0' '\n' | grep PROJECTS_ROOT

# Should show: PROJECTS_ROOT=/opt/tower/projects

# Test project update curl -X POST -k -u admin:password \ https://localhost/api/v2/projects/1/update/ | jq

# Check job launches curl -X POST -k -u admin:password \ https://localhost/api/v2/job_templates/1/launch/ | jq '.job'

# Verify project files are accessible ls -la /opt/tower/projects/ ls -la /opt/tower/job_output/ ```

Step 7: Fix Ansible Search Paths

Update Ansible's configuration to use new paths:

```bash # Update or create ansible.cfg in Tower home sudo vim /etc/tower/ansible.cfg

[defaults] library = /opt/tower/library:/usr/share/ansible/modules module_utils = /opt/tower/module_utils roles_path = /opt/tower/roles:/etc/tower/roles collections_path = /opt/tower/collections:/usr/share/ansible/collections inventory = /opt/tower/inventory

# Update environment variable in settings sudo vim /etc/tower/settings.py

AWX_ANSIBLE_CONFIG = '/etc/tower/ansible.cfg'

# Or via environment export ANSIBLE_CONFIG=/etc/tower/ansible.cfg export ANSIBLE_LIBRARY=/opt/tower/library export ANSIBLE_ROLES_PATH=/opt/tower/roles ```

Step 8: Migrate Data If Needed

Move actual data to new location:

```bash # Sync data from old to new location sudo rsync -avz --progress /var/lib/awx/projects/ /opt/tower/projects/ sudo rsync -avz --progress /var/lib/awx/job_output/ /opt/tower/job_output/

# Verify data integrity diff -rq /var/lib/awx/projects /opt/tower/projects || echo "Differences found"

# Update permissions sudo chown -R awx:awx /opt/tower/projects sudo chown -R awx:awx /opt/tower/job_output sudo chmod -R 755 /opt/tower/projects sudo chmod -R 755 /opt/tower/job_output

# Verify disk space df -h /opt/tower/ ```

Verification

Confirm all operations use the new path:

```bash # Check no references to old path remain grep -r "/var/lib/awx/projects" /etc/tower/ 2>/dev/null || echo "No old path references found"

# Verify project list shows correct paths curl -k -u admin:password https://localhost/api/v2/projects/ | \ jq '.results[] | {name, local_path}'

# All paths should start with /opt/tower/projects

# Test a full job execution curl -X POST -k -u admin:password \ https://localhost/api/v2/job_templates/1/launch/ | jq -r '.job' > /tmp/job_id.txt

# Check job completed successfully curl -k -u admin:password "https://localhost/api/v2/jobs/$(cat /tmp/job_id.txt)/" | \ jq '{status, job_explanation}'

# Verify job output stored in new location ls -la /opt/tower/job_output/job_$(cat /tmp/job_id.txt)/

# Test inventory sync curl -X POST -k -u admin:password \ https://localhost/api/v2/inventory_sources/1/update/ | jq '.status'

# Check logs for any path errors sudo grep -i "no such file|not found" /var/log/tower/*.log | tail -20 ```

  • [ansible-project-sync-fails-path-not-found](/articles/ansible-project-sync-fails-path-not-found)
  • [ansible-custom-modules-not-found](/articles/ansible-custom-modules-not-found)
  • [ansible-job-output-storage-migration](/articles/ansible-job-output-storage-migration)
  • [WordPress troubleshooting: Ansible Artifact Download Uses an Old Mi](ansible-artifact-download-uses-an-old-mirror-after-proxy-change)
  • [WordPress troubleshooting: Ansible Audit Trail Misses Events Under ](ansible-audit-trail-misses-events-under-burst-load)
  • [WordPress troubleshooting: Ansible Background Worker Gets Stuck in ](ansible-background-worker-stuck-in-a-retry-loop)
  • [WordPress troubleshooting: Ansible Backup Completes but Restore Fai](ansible-backup-completes-but-restore-fails-checksum-validation)
  • [WordPress troubleshooting: Ansible Batch Importer Duplicates Rows A](ansible-batch-importer-duplicates-rows-after-a-retry)

<script type="application/ld+json"> { "@context": "https://schema.org", "@type": "TechArticle", "headline": "WordPress troubleshooting: Ansible Runtime Still Reads the Old File", "description": "Learn how to fix Ansible Runtime Still Reads the Old File Path After a Directory Move. Professional WordPress troubleshooting solutions with step-by-step guidance. WP error fix, WordPress optimization, WP security, WordPress performance.", "url": "https://www.fixwikihub.com/ansible-runtime-still-reads-old-file-path-after-directory-move", "publisher": { "@type": "Organization", "name": "FixWikiHub", "url": "https://www.fixwikihub.com" }, "author": { "@type": "Person", "name": "FixWikiHub Editorial Team" }, "datePublished": "2026-02-12T16:09:42.913Z", "dateModified": "2026-02-12T16:09:42.913Z" } </script>