#1121878 Default::Packages::Compress=zstd does not work (Packges.zst without Packages is ignored)

Package:
apt
Source:
apt
Description:
commandline package manager
Submitter:
Trent W. Buck
Date:
2025-12-04 10:01:01 UTC
Severity:
normal
#1121878#5
Date:
2025-12-04 03:37:42 UTC
From:
To:
If an InRelease file lists Packages and Packages.zst, then
apt will download Packages.zst (good, normal).

If an InRelease File lists only Packages.zst, then
apt will silently ignore it and exit successfully.
This results in "apt update" appearing to work, but
none of the packages in the repo are available.

An attached transcript demonstrates the problem on sid.

This issue affects apt 3.1.12 (current sid) as at December 2025.
This issue affects apt 3.0.3 (current stable) as at December 2025.
This issue affects apt 1.8.4 (using xz instead of zst) as at February 2020.
This issue affects apt 1.4.9 (using xz instead of zst) as at February 2020.

I couldn't immediately work out where in the apt source code this issue is happening.
Is this intended behaviour, or a logic bug?
I speculate the current logic might be something like

    for each Packages (no suffix) in Release
        if it is for our architecture
            get url of best available compression
            download that url

and since there's no Packages (without suffix), the loop never runs at all.

It is not a big deal for me to continue doing

    Default::Packages::Compress    ". zstd";
    Default::Sources::Compress     ". zstd";
    Default::Contents::Compress    ". zstd";
    Default::Translation::Compress ". zstd";

instead of

    Default::Packages::Compress    "zstd";
    Default::Sources::Compress     "zstd";
    Default::Contents::Compress    "zstd";
    Default::Translation::Compress "zstd";

it is just confusing/irritating that Packages has to exist,
even though nothing ever downloads it.

#1121878#16
Date:
2025-12-04 07:13:08 UTC
From:
To:
A Release file always needs to list the decompressed file otherwise the file cannot be verified for correctness post extraction. The uncompressed file doesn't need to exist for that.

This is documented in the repository format specification

<https://wiki.debian.org/DebianRepository/Format?action=show&redirect=RepositoryFormat#MD5Sum.2C_SHA1.2C_SHA256>

The apt-ftparchive command does not handle decompressing files to calculate the decompressed checksum and ensure correctness across multiple compressors I suppose

#1121878#21
Date:
2025-12-04 07:33:08 UTC
From:
To:
OK, that makes sense.

I thought/hoped apt could validate the sha512 + length of Packages.zst
*before* decompression, and
therefore wouldn't need to also validate the sha512 + length of Packages
*after* decompression.

But if it did that and then made a mistake (checking neither in some
edge case scenario), the impact would be high.

So I think the existing behaviour (always checking the uncompressed
sum+length) is reasonable.

#1121878#26
Date:
2025-12-04 09:52:20 UTC
From:
To:
Am Thu, Dec 04, 2025 at 06:33:08PM +1100, schrieb Trent W. Buck:

It does, to e.g. protect against an attacker sending you a decompression
bomb or exploiting the decompression in some other fancy way.

As a way of checking that the decompression worked it does this, too.
(*mumbling something about 'Defense in depth'*).


It is also a handy way of keeping the checksum around for later.
A future apt run will skip downloading unchanged Packages files, but
it can only know that with the uncompressed hashes as the compressed
hashes are not stable.

If the repository offers pdiffs as a way of updating the indexes the
hash is also used to find the patch(es) we have to apply.

Both could be done by calculating the hash on the fly of course.

APT doesn't do it currently, but if we wanted to verify that what
we have on disk is really what we should have and not somehow
modified/corrupted we need a hash again (and stored there it can
not be tempered with easily) and the compressed hashes will produce
false positives if client and server use e.g. different (versions of)
(compression level options of) compressors.

Making this a feature request for apt-ftparchive to depose of the
uncompressed file (or not writing it to disc to begin with)
seems also reasonable – which Julian made of this now.


Best regards

David Kalnischkies