#1086639 apt: Mirrorlist files are always fully downloaded

Package:
apt
Source:
apt
Description:
commandline package manager
Submitter:
Ognyan Kulev
Date:
2024-11-03 08:45:01 UTC
Severity:
normal
#1086639#5
Date:
2024-11-02 18:29:23 UTC
From:
To:
Each `apt update` downloads fully the specified mirrorlist file, instead
of using If-Modified-Since: HTTP header to skip downloading the same
content.

Comparing it to other downloads, mirrorlist files are not downloaded
first into partial/ but anyway apt treats them as partial and requests
continuing the "partial" download. When this fails, the file is
downloaded completely. The expected behavior is requests to have
If-Modified-Since: HTTP header like other already downloaded files. A
sample session follows:

Fetching
mirror+https://fire.tower.3.bg/debian-mirrorlist.txt/dists/testing/InRelease
  to
/var/lib/apt/lists/partial/fire.tower.3.bg_debian-mirrorlist.txt_dists_testing_InRelease
  Queue is: mirror+https
Fetching https://fire.tower.3.bg/debian-mirrorlist.txt
  to /var/lib/apt/lists/auxfiles/fire.tower.3.bg_debian-mirrorlist.txt
  Queue is: https:fire.tower.3.bg
GET /debian-mirrorlist.txt HTTP/1.1
Host: fire.tower.3.bg
Range: bytes=1673-
If-Range: Sat, 02 Nov 2024 18:11:05 GMT
User-Agent: Debian APT-HTTP/1.3 (2.9.10)

Answer for: https://fire.tower.3.bg/debian-mirrorlist.txt
HTTP/1.1 200 OK
Server: nginx
Date: Sat, 02 Nov 2024 18:11:14 GMT
Content-Type: text/plain
Content-Length: 1673
Last-Modified: Sat, 02 Nov 2024 13:05:14 GMT
Connection: keep-alive
ETag: "6726238a-689"
Accept-Ranges: bytes


Best regards,
Ognyan
#1086639#10
Date:
2024-11-03 08:31:37 UTC
From:
To:
Hi (again),

Consecutive `apt update` invocations lead to alternating requests to the
mirrorlist location. In the original report is one of them: incorrect
If-Range date is responded with 200 OK and the full content. Here is a
sample of the other case of the alternation when If-Range is correct
date of last modification, so the HTTP server tries to satisfy the
Range: header. The file in auxfiles/ is updated with the current time
and in the next `apt update` this incorrect date will be used (like in
the original report).

Fetching https://fire.tower.3.bg/debian-mirrorlist.txt
  to /var/lib/apt/lists/auxfiles/fire.tower.3.bg_debian-mirrorlist.txt
  Queue is: https:fire.tower.3.bg
GET /debian-mirrorlist.txt HTTP/1.1
Host: fire.tower.3.bg
Range: bytes=1673-
If-Range: Sat, 02 Nov 2024 13:05:14 GMT
User-Agent: Debian APT-HTTP/1.3 (2.9.10)


Answer for: https://fire.tower.3.bg/debian-mirrorlist.txt
HTTP/1.1 416 Requested Range Not Satisfiable
Server: nginx
Date: Sun, 03 Nov 2024 08:22:26 GMT
Content-Type: text/html
Content-Length: 190
Connection: keep-alive
Content-Range: bytes */1673


Best regards,
Ognyan