Introduction

When you run ansible-tower-setup backup or use Tower's built-in backup functionality, the process completes successfully and creates a tarball. However, when you attempt to restore this backup to a new Tower instance or after a disaster recovery scenario, the restore fails with checksum validation errors. The backup file appears valid, the tarball extracts correctly, but the restore process rejects it as corrupted.

This issue occurs due to character encoding mismatches between the backup and restore environments, filesystem differences that alter file metadata, incomplete writes during backup creation, or timezone/locale differences affecting timestamp-based checksums.

Symptoms

The restore process fails with checksum mismatch errors:

``` TASK [backup_restore : Verify backup archive] ********** fatal: [localhost]: FAILED! => { "changed": false, "msg": "Checksum validation failed for /var/lib/awx/backups/tower-backup-2026-04-11.tar.gz. Expected: a1b2c3d4e5f6..., Got: f6e5d4c3b2a1..." }

TASK [backup_restore : Extract and validate backup] ******** fatal: [localhost]: FAILED! => {"msg": "Archive checksum does not match manifest"} ```

The Tower setup log shows:

bash
Checking integrity of backup archive...
/var/lib/awx/backups/tower-backup-2026-04-11.tar.gz: FAILED
sha256sum: WARNING: 1 computed checksum did NOT match
ERROR! Backup archive failed integrity check. Aborting restore.

When you manually verify the checksum:

```bash # On backup server sha256sum /var/lib/awx/backups/tower-backup-2026-04-11.tar.gz a1b2c3d4e5f6789012345678901234567890abcdef1234567890abcdef123456 tower-backup-2026-04-11.tar.gz

# After copying to restore server sha256sum tower-backup-2026-04-11.tar.gz f6e5d4c3b2a1098765432109876543210987654fedcba0987654fedcba098765 tower-backup-2026-04-11.tar.gz # Different checksum despite file copy! ```

The backup manifest shows correct entries:

bash
$ tar -tzf tower-backup-2026-04-11.tar.gz | head
./
./manifest.json
./database.sql
./credentials/
./credentials/1.json
./projects/

But extraction reveals encoding issues:

bash
$ tar -xzf tower-backup-2026-04-11.tar.gz
tar: credentials/15.json: Cannot open: Invalid argument
tar: Exiting with failure status due to previous errors

Common Causes

1. Character Encoding Mismatch in Credential Files

Tower credentials often contain passwords with special characters. When the backup system's locale differs from the restore system, character encoding changes alter the file content:

```bash # Backup system locale $ locale LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8

# Restore system locale $ locale LANG=C.UTF-8 # Different encoding for special characters ```

2. Filesystem Timestamp Precision

The backup checksum includes file metadata. Different filesystems (ext4 vs XFS vs NFS) have different timestamp precision:

bash
# ext4 supports nanosecond precision
# XFS may truncate to microseconds
# This changes the tarball content and thus the checksum

3. Incomplete Write During Backup

The backup completes but the tarball is written asynchronously. If the system is under load or the backup drive is slow, the file may not be fully flushed:

python
# Backup code may not call fsync
with tarfile.open(backup_path, 'w:gz') as tar:
    tar.add('/var/lib/awx/')
# File handle closes but data may still be in OS cache

4. Different tar or gzip Versions

Different versions of tar and gzip produce different outputs for the same input:

```bash # Backup server $ tar --version tar (GNU tar) 1.34

# Restore server $ tar --version tar (GNU tar) 1.30 # Different compression algorithms = different output ```

5. Windows Line Endings in Manifest

If the backup was created or modified on Windows, line endings may be converted:

bash
$ file manifest.json
manifest.json: JSON data, with CRLF line terminators
# Should be LF only

Step-by-Step Fix

Step 1: Diagnose the Checksum Failure Source

Determine where the corruption occurs:

```bash # Generate checksum on backup server immediately after creation sha256sum /var/lib/awx/backups/tower-backup-*.tar.gz > /tmp/checksums.sha256

# Copy checksums with backup cp /tmp/checksums.sha256 /var/lib/awx/backups/

# Compare after transfer to restore server sha256sum -c checksums.sha256 # tower-backup-2026-04-11.tar.gz: FAILED ```

Check for encoding differences:

```bash # Check locale on both systems locale

# Check for binary differences cmp -l backup-server.tar.gz restore-server.tar.gz | head -20 ```

Verify tarball integrity:

```bash # Test tarball can be read gzip -t tower-backup-2026-04-11.tar.gz

# List contents without extracting tar -tzf tower-backup-2026-04-11.tar.gz > /dev/null echo "Exit code: $?" ```

Step 2: Re-create Backup with Encoding Control

Run backup with explicit locale and encoding settings:

```bash # Set locale explicitly before backup export LANG=en_US.UTF-8 export LC_ALL=en_US.UTF-8

# Run Tower backup ./setup.sh -b

# Or for manual backup ansible-playbook -i inventory backup.yml \ -e "backup_locale=en_US.UTF-8" ```

Create a more robust backup script:

```bash #!/bin/bash # /usr/local/bin/tower-backup-robust.sh

set -euo pipefail

BACKUP_DIR="/var/lib/awx/backups" DATE=$(date +%Y-%m-%d-%H%M%S) BACKUP_FILE="${BACKUP_DIR}/tower-backup-${DATE}.tar.gz"

# Ensure consistent locale export LANG=en_US.UTF-8 export LC_ALL=en_US.UTF-8

# Create backup with explicit settings tar --create \ --gzip \ --file="${BACKUP_FILE}" \ --directory=/var/lib/awx \ --exclude='*.log' \ --exclude='*.pid' \ --format=posix \ --pax-option=exthdr.name=%d/PaxHeaders/%f,delete=atime,delete=ctime \ .

# Force sync to disk sync fsync "${BACKUP_FILE}"

# Generate checksums sha256sum "${BACKUP_FILE}" > "${BACKUP_FILE}.sha256"

# Verify backup is readable gzip -t "${BACKUP_FILE}" || { echo "ERROR: Backup failed integrity test" rm -f "${BACKUP_FILE}" "${BACKUP_FILE}.sha256" exit 1 }

echo "Backup created: ${BACKUP_FILE}" cat "${BACKUP_FILE}.sha256" ```

Step 3: Modify Tower Backup Playbook for Better Portability

Update the backup playbook to handle encoding:

```yaml # backup_tower.yml - Enhanced backup playbook - name: Create Tower backup with validation hosts: localhost vars: backup_dir: /var/lib/awx/backups backup_name: "tower-backup-{{ ansible_date_time.iso8601_basic_short }}"

tasks: - name: Ensure consistent locale set_fact: backup_env: LANG: en_US.UTF-8 LC_ALL: en_US.UTF-8 TZ: UTC

  • name: Create backup directory
  • file:
  • path: "{{ backup_dir }}"
  • state: directory
  • mode: '0750'
  • name: Dump PostgreSQL database
  • shell: |
  • pg_dump -U awx -Fc awx > {{ backup_dir }}/database.dump
  • environment: "{{ backup_env }}"
  • become_user: awx
  • name: Export Tower objects to JSON
  • shell: |
  • awx-manage export_objects --all > {{ backup_dir }}/tower_objects.json
  • environment: "{{ backup_env }}"
  • become_user: awx
  • name: Create tarball with consistent settings
  • archive:
  • path:
  • - "{{ backup_dir }}/database.dump"
  • - "{{ backup_dir }}/tower_objects.json"
  • - /var/lib/awx/projects/
  • - /var/lib/awx/job_output/
  • dest: "{{ backup_dir }}/{{ backup_name }}.tar.gz"
  • format: gz
  • environment: "{{ backup_env }}"
  • name: Force filesystem sync
  • command: sync
  • changed_when: true
  • name: Generate SHA256 checksum
  • stat:
  • path: "{{ backup_dir }}/{{ backup_name }}.tar.gz"
  • checksum_algorithm: sha256
  • register: backup_stat
  • name: Write checksum file
  • copy:
  • content: "{{ backup_stat.stat.checksum }} {{ backup_name }}.tar.gz\n"
  • dest: "{{ backup_dir }}/{{ backup_name }}.tar.gz.sha256"
  • name: Verify backup can be extracted
  • command: tar -tzf {{ backup_dir }}/{{ backup_name }}.tar.gz
  • changed_when: false
  • name: Verify checksum matches
  • shell: |
  • cd {{ backup_dir }}
  • sha256sum -c {{ backup_name }}.tar.gz.sha256
  • changed_when: false
  • `

Step 4: Restore with Checksum Bypass (If Necessary)

If you must restore a backup with mismatched checksum but verified content:

```bash # First, verify the tarball can be extracted tar -tzf tower-backup-2026-04-11.tar.gz > /dev/null && echo "Archive is valid"

# Extract and verify contents mkdir -p /tmp/backup-test tar -xzf tower-backup-2026-04-11.tar.gz -C /tmp/backup-test ls -la /tmp/backup-test/

# If contents are valid, modify the restore playbook to skip checksum ./setup.sh -e "skip_checksum_validation=true" ```

Or modify the restore playbook directly:

```yaml # In the restore playbook, add conditional - name: Verify backup archive stat: path: "{{ backup_file }}" checksum_algorithm: sha256 register: backup_stat when: not skip_checksum_validation | default(false)

  • name: Validate checksum
  • fail:
  • msg: "Checksum validation failed for {{ backup_file }}"
  • when:
  • - not skip_checksum_validation | default(false)
  • - backup_stat.stat.checksum != expected_checksum
  • `

Step 5: Implement Backup Verification Pipeline

Create a verification job template in Tower:

```yaml # verify_backup.yml - name: Verify Tower backup integrity hosts: localhost vars: backup_file: "{{ backup_path | default('/var/lib/awx/backups/latest.tar.gz') }}"

tasks: - name: Check if backup file exists stat: path: "{{ backup_file }}" register: backup_stat

  • name: Fail if backup missing
  • fail:
  • msg: "Backup file not found: {{ backup_file }}"
  • when: not backup_stat.stat.exists
  • name: Verify GZIP integrity
  • command: gzip -t {{ backup_file }}
  • changed_when: false
  • register: gzip_test
  • name: Verify TAR integrity
  • command: tar -tzf {{ backup_file }}
  • changed_when: false
  • register: tar_test
  • name: Check for checksum file
  • stat:
  • path: "{{ backup_file }}.sha256"
  • register: checksum_file
  • name: Verify SHA256 checksum
  • shell: |
  • cd {{ backup_file | dirname }}
  • sha256sum -c {{ backup_file | basename }}.sha256
  • changed_when: false
  • when: checksum_file.stat.exists
  • register: checksum_test
  • name: Extract test sample
  • shell: |
  • mkdir -p /tmp/backup_verify
  • tar -xzf {{ backup_file }} -C /tmp/backup_verify manifest.json
  • cat /tmp/backup_verify/manifest.json
  • changed_when: false
  • register: manifest
  • name: Report backup status
  • debug:
  • msg:
  • - "Backup file: {{ backup_file }}"
  • - "Size: {{ backup_stat.stat.size }} bytes"
  • - "GZIP valid: {{ gzip_test.rc == 0 }}"
  • - "TAR valid: {{ tar_test.rc == 0 }}"
  • - "Checksum valid: {{ checksum_test.rc | default('N/A') }}"
  • - "Manifest: {{ manifest.stdout | from_json | default('Invalid JSON') }}"
  • `

Verification

Test the backup and restore process:

```bash # Create test backup ./tower-backup-robust.sh

# Verify checksums match sha256sum -c /var/lib/awx/backups/*.sha256

# Copy to restore server and verify scp tower-backup-*.tar.gz restore-server:/tmp/ ssh restore-server "sha256sum /tmp/tower-backup-*.tar.gz"

# Compare checksums diff <(sha256sum tower-backup-*.tar.gz | cut -d' ' -f1) \ <(ssh restore-server "sha256sum /tmp/tower-backup-*.tar.gz" | cut -d' ' -f1) # Should show no difference

# Test restore ./setup.sh -r -e "backup_file=/tmp/tower-backup-latest.tar.gz" ```

Verify encoding consistency:

bash
# Check locale on both systems match
ssh backup-server "locale" > /tmp/backup-locale.txt
locale > /tmp/restore-locale.txt
diff /tmp/backup-locale.txt /tmp/restore-locale.txt
  • [ansible-cleanup-job-removes-shared-files-still-needed-by-runtime](/articles/ansible-cleanup-job-removes-shared-files-still-needed-by-runtime) - File cleanup affecting backups
  • [ansible-migration-holds-the-main-table-longer-than-the-deployment-window](/articles/ansible-migration-holds-the-main-table-longer-than-the-deployment-window) - Database migration issues
  • [ansible-file-permission-reset-after-base-image-change](/articles/ansible-file-permission-reset-after-base-image-change) - Permission issues after restore
  • [WordPress troubleshooting: Ansible Artifact Download Uses an Old Mi](ansible-artifact-download-uses-an-old-mirror-after-proxy-change)
  • [WordPress troubleshooting: Ansible Audit Trail Misses Events Under ](ansible-audit-trail-misses-events-under-burst-load)
  • [WordPress troubleshooting: Ansible Background Worker Gets Stuck in ](ansible-background-worker-stuck-in-a-retry-loop)
  • [WordPress troubleshooting: Ansible Batch Importer Duplicates Rows A](ansible-batch-importer-duplicates-rows-after-a-retry)
  • [WordPress troubleshooting: Ansible Batch Writer Commits Partial Res](ansible-batch-writer-commits-partial-results-before-final-validation)

<script type="application/ld+json"> { "@context": "https://schema.org", "@type": "TechArticle", "headline": "WordPress troubleshooting: Ansible Backup Completes but Restore Fai", "description": "Learn how to fix Ansible Backup Completes but Restore Fails Checksum Validation. Professional WordPress troubleshooting solutions with step-by-step guidance. WP error fix, WordPress optimization, WP security, WordPress performance.", "url": "https://www.fixwikihub.com/ansible-backup-completes-but-restore-fails-checksum-validation", "publisher": { "@type": "Organization", "name": "FixWikiHub", "url": "https://www.fixwikihub.com" }, "author": { "@type": "Person", "name": "FixWikiHub Editorial Team" }, "datePublished": "2026-02-12T17:32:23.691Z", "dateModified": "2026-02-12T17:32:23.691Z" } </script>