Dear Maintainer,
when doing `apt update` with apt-cacher-ng proxy configured in apt, often am getting
error "E: Failed to fetch .../main/source/Sources.xz Hash Sum mismatch". then apt
exits with an error: "E: Failed to fetch .../main/binary-armhf/Packages.xz E: Some
index files failed to download. They have been ignored, or old ones used instead."
Once it is encountered, any subsequent `apt update` fails with same error, after again
downloading the affected file(s) from the proxy/cache.
Bypassing the proxy (e.g., `sudo apt update -o Acquire::http::proxy=''`) is successful.
Inspecting /var/lib/apt/lists/partial/ there will be a ..._Sources.xz.FAILED. Manually
replacing this file (Sources.xz) with one downloaded directly from source will also
allow the `apt update` to be successful.
re: apt client, occurs on Raspberry Pi OS (both buster and bullseye), Debian 11 (bullseye,
amd64) and Debian testing.
occuring on recent dist versions of apt-cacher-ng (both bullseye and bookworm) running
from VMs on amd64 host.
Doing some investigation, found some interesting results:
- checksum from /var/lib/apt/lists/partial/ .FAILED file matched the reported mismatching
checksum reported in apt update error
- checksum from using wget to download directly from source matched the expected checksum
- inpecting file in /var/cache/apt-cacher-ng/... on apt-cacher-ng host server matches
the EXPECTED checksum, meaning that it doesn't seem to be a race
- file lengths are the same (always)
- binary diff of original compressed files shows huge chunks of the expected file vs.
mismatched file are common. and then large continuous chunks mismatch, and then re-sync
(are same in both) for huge spans and this repeats.
- observed this by generating text representation of the binary/compressed generated
using xxd and side-by-side diffs using sdiff
EXAMPLE Mismatch Error message
E: Failed to fetch http://raspbian.raspberrypi.org/raspbian/dists/bullseye/main/source/Sources.xz Hash Sum mismatch
Hashes of expected file:
- Filesize:12223504 [weak]
- SHA256:ed08efd74766e9d76d73f00b200f37ce41e65a51de927114a1e00a3e1f57cc7c
- SHA1:6399b9bc363c2a0d50ede8c871d92bc6d235c9bb [weak]
- MD5Sum:3148fbb97feeb4b5fa6b7f1646ab701d [weak]
Hashes of received file:
- SHA256:89f42ada72c8c8157b78b8ed173f0feedc9d788e8bcf543ddc447dbda6bf6b26
- SHA1:d835932deab7abe06bd0ee3edbf3a7fea415b2aa [weak]
- MD5Sum:af435d409c99ba2950364c21886ea710 [weak]
- Filesize:12223504 [weak]
Last modification reported: Thu, 11 Nov 2021 23:20:36 +0000
Release file created at: Thu, 11 Nov 2021 23:22:43 +0000
E: Failed to fetch http://raspbian.raspberrypi.org/raspbian/dists/bullseye/main/binary-armhf/Packages.xz
E: Some index files failed to download. They have been ignored, or old ones used instead.