Barman woes on OVH

So, due to a cascade of Barman’s errors and corrupted backups due to running out of backup space, I had to pretty much clear out my Barman backup storage.

After resetting everything, however, I was not able to make a full backup, because Barman was not able to receive write-ahead logs from PostgreSQL.

2025-06-04 22:00:21,209 [523687] barman.cli ERROR: [Errno 5] Input/output error: '/backup/barman/pg/wals/00000001000000CD/00000001000000CD00000009.tmp'
See log file for more details.
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/barman/cli.py", line 2390, in main
    args.func(args)
  File "/usr/lib/python3/dist-packages/barman/cli.py", line 1600, in archive_wal
    server.archive_wal()
  File "/usr/lib/python3/dist-packages/barman/server.py", line 2651, in archive_wal
    self.backup_manager.archive_wal(verbose)
  File "/usr/lib/python3/dist-packages/barman/backup.py", line 847, in archive_wal
    archiver.archive(verbose)
  File "/usr/lib/python3/dist-packages/barman/wal_archiver.py", line 213, in archive
    self.archive_wal(compressor, wal_info)
  File "/usr/lib/python3/dist-packages/barman/wal_archiver.py", line 356, in archive_wal
    shutil.copystat(src_file, tmp_file)
  File "/usr/lib/python3.12/shutil.py", line 388, in copystat
    _copyxattr(src, dst, follow_symlinks=follow)
  File "/usr/lib/python3.12/shutil.py", line 338, in _copyxattr
    os.setxattr(dst, name, value, follow_symlinks=follow_symlinks)
OSError: [Errno 5] Input/output error: '/backup/barman/pg/wals/00000001000000CD/00000001000000CD00000009.tmp'

Input/output error? That’s odd, but the stacktrace tells us a lot, as from the function name (os.setxattr), we can deduce that it’s trying to set xattrs on our WAL files. The underlying storage for my backups is OVH’s Backup Storage, accessible over NFS. And NFS, for most of its life, was not able to support xattrs even if the underlying filesystem does. The support has been added to NFS 4.2, while OVH (still) uses 4.1.

So, how to fix this?

Initially, I thought of downgrading Barman, because things had worked before. But that did not help, and so I had to go digging into the source code (which was painful as I am not a Python guy).

# Perform the real filesystem operation with the xlogdb lock taken.
            # This makes the operation atomic from the xlogdb file POV
            with self.server.xlogdb("a") as fxlogdb:
                # If the content has changed, it means the file was either compressed
                # or encrypted or both. In this case, we need to update its metadata
                if content_changed:
                    shutil.copystat(src_file, current_file)
                    stat = os.stat(current_file)
                    wal_info.size = stat.st_size

So, if content_changed is true, we use copystat from shutil.py, which copies xattrs from the original file.

# If the bits of the file has changed e.g. due to compression or encryption
            content_changed = False
            # Compress the file if not already compressed
            if compressor and not wal_info.compression:
                compressor.compress(src_file, tmp_file)
                files_to_remove.append(current_file)
                current_file = tmp_file
                content_changed = True
                wal_info.compression = compressor.compression
            # Encrypt the file
            if encryption:
                encrypted_file = encryption.encrypt(current_file, dst_dir)
                files_to_remove.append(current_file)
                current_file = encrypted_file
                wal_info.encryption = encryption.NAME
                content_changed = True

Ah, so therein lies the rub: Barman assumes that it needs to do this if the content is either being compressed or encrypted. And it just so happens that I’ve also enabled GZIP compression so as not to run out of space again. Well, we have to deal with this the old-fashioned way (by lowering the retention policy). After disabling compression, Barman was able to make backups again.

Hope this helps someone, because it sure as fuck would’ve helped me.