Since months i have an error during the apt update phase, always on
experimental/non-free. Initially i thought it was a temporary mirror
problem, because changing the repository the error went away.
But when in the next days i was doing another update, the error came back,
even with the changed mirror.

It happens on at least two machine of mine, an i386 and an amd64 arch, with
substantially the same sources.list and i think it started to show this
behaviour at the same time.
And it happens with aptitude too.

Apart from sources.list, i think i've never changed nothing from default
apt setup.

Try these steps, to me are always reproducible:
- Take my sources.list.
- If you are currently using the same mirror like mine, change it.
- apt update
- Look that no error are displayed: everything is correctly updated.
- Rerun: apt update
- Here i see an "Err" line on experimental/non-free Packages.

Here is my output:
-------------------------
root@etna:~# apt update
Get:1 http://httpredir.debian.org unstable InRelease [204 kB]
Get:2 http://httpredir.debian.org experimental InRelease [162 kB]
Get:3 http://httpredir.debian.org unstable/contrib Translation-en [42,1 kB]
Get:4 http://httpredir.debian.org unstable/main Translation-en [4.878 kB]
Get:5 http://httpredir.debian.org unstable/non-free Translation-en [76,4 kB]
Get:6 http://httpredir.debian.org experimental/contrib Translation-en [4.527 B]
Get:7 http://httpredir.debian.org experimental/main Translation-en [429 kB]
Get:8 http://httpredir.debian.org unstable/main amd64 Packages [7.189 kB]
Get:9 http://httpredir.debian.org experimental/non-free Translation-en [10,2 kB]
Get:10 http://httpredir.debian.org experimental/contrib amd64 Packages [6.144 B]
Get:11 http://httpredir.debian.org experimental/non-free amd64 Packages [13,9 kB]
Get:12 http://httpredir.debian.org unstable/non-free amd64 Packages [89,2 kB]
Get:13 http://httpredir.debian.org unstable/contrib amd64 Packages [54,1 kB]
Get:14 http://httpredir.debian.org experimental/main amd64 Packages [737 kB]
Fetched 13,9 MB in 24s (568 kB/s)
Reading package lists... Done
Building dependency tree
Reading state information... Done
34 packages can be upgraded. Run 'apt list --upgradable' to see them.
root@etna:~#
root@etna:~# apt update
Get:1 http://httpredir.debian.org unstable InRelease [204 kB]
Hit http://httpredir.debian.org experimental InRelease
Get:2 http://httpredir.debian.org experimental/contrib amd64 Packages/DiffIndex [7.819 B]
Get:3 http://httpredir.debian.org experimental/main amd64 Packages/DiffIndex [7.819 B]
Get:4 http://httpredir.debian.org experimental/non-free amd64 Packages/DiffIndex [7.819 B]
Get:5 http://httpredir.debian.org experimental/contrib Translation-en/DiffIndex [7.819 B]
Get:6 http://httpredir.debian.org experimental/main Translation-en/DiffIndex [7.819 B]
Get:7 http://httpredir.debian.org experimental/non-free Translation-en/DiffIndex [7.819 B]
Get:8 http://httpredir.debian.org unstable/main amd64 Packages/DiffIndex [7.876 B]
Get:9 http://httpredir.debian.org unstable/non-free amd64 Packages/DiffIndex [7.819 B]
Get:10 http://httpredir.debian.org unstable/contrib amd64 Packages/DiffIndex [7.819 B]
Get:11 http://httpredir.debian.org unstable/contrib Translation-en/DiffIndex [7.819 B]
Get:12 http://httpredir.debian.org unstable/main Translation-en/DiffIndex [7.876 B]
Get:13 http://httpredir.debian.org unstable/non-free Translation-en/DiffIndex [7.819 B]
Err http://httpredir.debian.org experimental/non-free amd64 Packages

Hit http://httpredir.debian.org experimental/non-free amd64 Packages
Fetched 298 kB in 4s (66,2 kB/s)
Reading package lists... Done
Building dependency tree
Reading state information... Done
34 packages can be upgraded. Run 'apt list --upgradable' to see them.
root@etna:~#
-------------------------

So when you change mirror and update for the first time you get no error.
And that's why in the past i thought was a mirror problem: because changing
mirror the error apparently went away.
But on subsequently updates the error is always present.

Cesare.

Hi,

Cesare Leonardi wrote:

Same here.

Dito.

Also the http redirection is not the cause as I get this with using
plain http on ftp.ch.debian.org:

Err http://ftp.ch.debian.org experimental/non-free amd64 Packages

I though currently can't reproduce it on i386.

		Regards, Axel

Hi,
[…]
apt update -o Debug::Acquire::http=1

(This enables a lot of output resembling the messages send between
server(s) and the apt client(s). Probably best to redirect the output to
a file with e.g.:  2>&1 | tee apt-update.log
as the different interactions happen in parallel, so the output can be
a confusing mix of stuff on first look).

We are working in /experimental on revamping the acquire system at large
which is what is behind downloading stuff and shared between all
package managers based on libapt, so apt and aptitude do exactly the
same here.

I am in particular working currently in my branch on the behavior of
getting a 'Hit' for a (In)Release(.gpg) file currently, which is what
you got in your example and likely most of the time as non-free
experimental isn't changing a lot.

Note that this Err isn't critical, apt happily recovers from many
'errors' as getting them is "normal". apt implements an HTTP1.1 client
which tries to use many features of HTTP1.1, but many servers running on
mirrors implement only a subset of the features. And many different
subsets. Not to mention that some downright refuse to implement them
correctly (one of the sillier examples is e.g. #778375 or pipelining, my
personal favorite). Unfortunately the internet isn't implemented based
on specs, but based on what popular browsers use and support (which
isn't much – its pretty ironic that our selfwritten test webserver
supports more of HTTP1.1 than most 'real' webservers. And its only
getting worse)…


Usually you are not told about such errors, but it looks like one
slipped through here. I haven't looked at the 'old' source, but
I have a suspicion, which the debug output hopefully confirms:

The server is supporting Range-requests, but not If-Range and hence
answers with a 416, but not with Content-Range set. I at least have
a faint memory of encountering such a server recently with these
symptoms.


So, summary: If the debug output isn't proofing me horribly wrong
(possible), I am going to close this bugreport (if unreproducible with
our forthcoming new acquire system) as this is a 'display issue' and the
fix is basically our 'webservers are idiots' rewrite^Wrework… ;)

haven't really tried for some time, the doubt started to arise and since
your reply i've run some tests.
I confirm what Axel said: the error doesn't show under i386. And i've
verified also on another i386 PCs (up to date Sid).

Then i've verified with two KVM virtual machine, both with updated Sid,
same sources.list, but one i386 and one amd64: i can see the error only
on amd64.

To be honest, now, while writing this mail, on the first i386 PC tested,
i get error on experimental/main, but only using httpredir in
sources.list and only in the first run after changing mirror. Don't know
what to think...

Cesare.

- 2 from i386 and 2 from amd64
- 2 for the first run, after changing mirror (run1) and 2 for the next
run (run2).

Under i386 i see that in the second run are searched only diff files,
while under amd64, only for experimental/main, are searched Packages.bz2
and Packages.xz: the first one generates the error, because doesn't exist.

Thank you very much for sharing information about the development of apt.
[...]
 > Usually you are not told about such errors, but it looks like one
 > slipped through here.

In fact in the first run i see a lot of error about apt searching for
bz2 files first (that are missing), then for xz (ok).

To me it doesn't look the case.

Cesare.