#975903 apt-get update fails to (permanently) download index files after time skew

Package:
apt
Source:
apt
Description:
commandline package manager
Submitter:
Philipp Hahn
Date:
2020-11-26 12:09:12 UTC
Severity:
normal
#975903#5
Date:
2020-11-26 11:58:06 UTC
From:
To:
Suspending a VM (Qemu/KVN) to disk while it is running also saves the
current time, which is restored on resume. If NTP does not work quickly
enough to fix that on resume, the VM lives in the past.
Running `apt-get update` will then print something like this:
The file itself is considered valid (correct signature, ...), but due to
the time skew the referenced `Packages*` and `Sources*` files are *NOT*
downloaded.

When next you invoke `apt-get update` again - even after fixing the time
and thus moving the VM to current wall clock - nothing gets updated.

`apt` find the valid `Release` file and checks it with the server.
If there is an updated version, everything works as expected.

But if there is no updated version, `apt` skips the following steps to
download the index files as it thinks, that they are still valid from
the *previous* download, which never happened.

This can be "fixed" by deleting the files `/var/lib/apt/lists/*Release`,
which forces `apt` to download thoses files again; the "changed"
`Release` file then triggeres the download of the changed index files,
too.


Reproducer:
  /etc/init.d/ntp stop
  date -s '2020-11-01'
  truncate -s 0 /var/lib/apt/lists/*_Packages
  touch /var/lib/apt/lists/*_*
  apt-get update -o Debug::Acquire::http=true
  > If-Modified-Since: Sat, 31 Oct 2020 23:00:10 GMT
  < HTTP/1.1 200 OK
  < Date: Thu, 26 Nov 2020 11:35:30 GMT
  < Last-Modified: Thu, 26 Nov 2020 11:16:48 GMT
  # will complain about times in the future
  ls -gGh /var/lib/apt/lists/*_*
  # will list `Packages*` files with size 0
  ntpdate 1.debian.pool.ntp.org
  apt-get update -o Debug::Acquire::http=true
  ls -gGh /var/lib/apt/lists/*_*
  # `Release` files will be downloaded again
  # `Packages*` files still has size 0

To me it looks like `apt`
1. gets an "HTTP 200 OK",
2. downloads the (unchagned) file,
3. but sees that it is the same as the locall file,
4. skips downloadthe the index files.

I haven't yet checked the code, but on first impulse `apt` should not
store the `Release` file if it is considered invalid due to time skew.

Or always try to download the referenced index files if the `Release`
file was downloaded, even when it remains unchanged to the previous
local version. Only on a "HTTP 304 Not Modified" should `apt` optimize
the index files download.


This issue can be mitigated by setting `Acquire::Max-FutureTime` to
a large value.