#1002491 ZFS transparent compression triggers spurious "Trailing allocated space"

Package:
apt-cacher-ng
Source:
apt-cacher-ng
Description:
caching proxy server for software repositories
Submitter:
"Trent W. Buck"
Date:
2021-12-23 07:42:04 UTC
Severity:
normal
#1002491#5
Date:
2021-12-23 07:38:02 UTC
From:
To:
NOTE: I notice "c) never truncating package files, only appending" in
https://salsa.debian.org/blade/apt-cacher-ng/-/blob/debian/sid/ChangeLog
      So this issue might ALREADY be fixed in testing/unstable.
      I have not tested this experimentally yet (sorry).

      Hopefully this bug report at least helps other Debian 11
      acng+ZFS users understand what's going on.

When /var/cache/apt-cacher-ng is a ZFS filesystem with transparent compression,
the space saving (or loss!) is visible on the command line with du --apparent-size:

    root@odin:~# ls -l /var/cache/apt-cacher-ng/debrep/dists/experimental/main/source/Sources.xz
    -rw-r--r-- 1 apt-cacher-ng apt-cacher-ng 342636 Dec 23 17:58 /var/cache/apt-cacher-ng/debrep/dists/experimental/main/source/Sources.xz

    root@odin:~# du --block-size=512 --apparent-size /var/cache/apt-cacher-ng/debrep/dists/experimental/main/source/Sources.xz
    740     /var/cache/apt-cacher-ng/debrep/dists/experimental/main/source/Sources.xz

    root@odin:~# du --block-size=512                 /var/cache/apt-cacher-ng/debrep/dists/experimental/main/source/Sources.xz
    786     /var/cache/apt-cacher-ng/debrep/dists/experimental/main/source/Sources.xz

On a system WITHOUT transparent compression,
these would normally report the same size (except for sparse files).

I *think* this difference causes apt-cacher-ng to mis-detect a file as being the wrong size:

    Trailing allocated space on /var/cache/apt-cacher-ng/debrep/dists/experimental/main/source/Sources.xz (786 blocks, expected: ~740), will be trimmed later

I only get 187 warnings though /var/cache/apt-cacher-ng has 20691 files.
I think this makes sense because most files in /var/cache/apt-cacher-ng are
already strongly compressed (e.g. Packages.xz or LZMA2's .deb).

"zfs get compressratio" supports this theory:

    COMPRESS        RATIO  USEDDS  NAME
    zstd            1.04x   7.70G  rpool/var/cache/apt-cacher-ng
    zstd            3.86x   3.02G  rpool
    zstd            5.33x    571M  rpool/var/log/journal

On this basis, a reasonable workaround is to simply disable
transparent compression on apt-cacher-ng's filesystem, e.g.

    zfs set compression=off rpool/var/cache/apt-cacher-ng

This will not affect files already written to disk, so
you'll still see trailing space warnings for existing files.
(Just delete them and let apt-cacher-ng re-fetch them, I guess.)