Running apt-get update on a system at the same time with other apt commands,
causes apt to resolve package dependencies and policies inconsistently.

Behavior causes race conditions during package cache update, resulting in altered
candidate versions and you will end up with unwanted versions being installed (eg.
when running 'apt-get upgrade -y’ or working with unattended-upgrades).

Current stable version in Jessie - 1.0.9.8 is also affected.

IMO, this should either be an atomic file/directory move, once package files where
downloaded successfully, or apt-get update should make use of locking as well.

We saw this issue affecting multiple apt commands and actions:

- apt-get upgrade
- apt-get dist-upgrade
- apt-cache policy
- aptitude update
- apt-get update

APT bug #717679 is probably related.

Reproduce:

Make sure your package is available in more than one APT repository (eg. dotdeb,
backports) and run apt-cache policy in a loop:

 while [ true ] ; do apt-cache policy augeas-lenses ; sleep .5 ; done

Now, while the above is running, update your package cache:

 apt-get update

You will see dependencies changing back and forth, depending on the current cache
state. Output:

augeas-lenses:
 Installed: 0.10.0-1
 Candidate: 0.10.0-1
 Version table:
    1.2.0-0.2~bpo70+2 0
       100 http://mirror.ono.at/debian/ wheezy-backports/main amd64 Packages
*** 0.10.0-1 0
       500 http://mirror.ono.at/debian/ wheezy/main amd64 Packages
       100 /var/lib/dpkg/status
augeas-lenses:
 Installed: 0.10.0-1
 Candidate: 1.2.0-0.2~bpo70+2
 Version table:
    1.2.0-0.2~bpo70+2 0
       100 http://mirror.ono.at/debian/ wheezy-backports/main amd64 Packages
*** 0.10.0-1 0
       100 /var/lib/dpkg/status
augeas-lenses:
 Installed: 0.10.0-1
 Candidate: 0.10.0-1
 Version table:
    1.2.0-0.2~bpo70+2 0
       100 http://mirror.ono.at/debian/ wheezy-backports/main amd64 Packages
*** 0.10.0-1 0
       500 http://mirror.ono.at/debian/ wheezy/main amd64 Packages
       100 /var/lib/dpkg/status
E: Problem renaming the file /var/cache/apt/pkgcache.bin.BCQoX0 to /var/cache/apt/pkgcache.bin - rename (2: No such file or directory)
W: You may want to run apt-get update to correct these problems
augeas-lenses:
 Installed: 0.10.0-1
 Candidate: 0.10.0-1
 Version table:
    1.2.0-0.2~bpo70+2 0
       100 http://mirror.ono.at/debian/ wheezy-backports/main amd64 Packages
*** 0.10.0-1 0
       500 http://mirror.ono.at/debian/ wheezy/main amd64 Packages
       100 /var/lib/dpkg/status

Same thing while running 'apt-get upgrade -s' in a loop, and starting 'apt-get update' at the same time, results in:

Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages will be upgraded:
 haproxy
1 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Inst haproxy [1.5.8-2~bpo70+1] (1.5.8-3~bpo70+1 Debian Backports:/wheezy-backports [amd64])
Conf haproxy (1.5.8-3~bpo70+1 Debian Backports:/wheezy-backports [amd64])
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages have been kept back:
 iproute libaugeas-ruby1.8 nagios-plugins-basic nagios-plugins-common nagios-plugins-standard openssh-client
 openssh-server python-debian vim-common vim-nox vim-runtime vim-tiny
The following packages will be upgraded:
 augeas-lenses dmidecode git git-core git-man haproxy libaugeas0 libgeoip1 libgpg-error0 libp11-kit0 libxapian22
 tar tmux
13 upgraded, 0 newly installed, 0 to remove and 12 not upgraded.
Inst tar [1.26+dfsg-0.1] (1.27.1-1~bpo70+1 Debian Backports:/wheezy-backports [amd64])
Conf tar (1.27.1-1~bpo70+1 Debian Backports:/wheezy-backports [amd64])
Inst libgpg-error0 [1.10-3.1] (1.12-0.2~bpo70+1 Debian Backports:/wheezy-backports [amd64])
Inst libp11-kit0 [0.12-3] (0.20.7-1~bpo70+1 Debian Backports:/wheezy-backports [amd64])
Inst augeas-lenses [0.10.0-1] (1.2.0-0.2~bpo70+2 Debian Backports:/wheezy-backports [all])
Inst libaugeas0 [0.10.0-1] (1.2.0-0.2~bpo70+2 Debian Backports:/wheezy-backports [amd64])
Inst libgeoip1 [1.4.8+dfsg-3] (1.6.2-4~bpo70+1 Debian Backports:/wheezy-backports [amd64])
Inst tmux [1.6-2] (1.9-6~bpo70+1 Debian Backports:/wheezy-backports [amd64])
Inst dmidecode [2.11-9] (2.12-2~bpo70+1 Debian Backports:/wheezy-backports [amd64])
Inst git [1:1.7.10.4-1+wheezy1] (1:1.9.1-1~bpo70+2 Debian Backports:/wheezy-backports [amd64]) []
Inst git-man [1:1.7.10.4-1+wheezy1] (1:1.9.1-1~bpo70+2 Debian Backports:/wheezy-backports [all])
Inst git-core [1:1.7.10.4-1+wheezy1] (1:1.9.1-1~bpo70+2 Debian Backports:/wheezy-backports [all])
Inst haproxy [1.5.8-2~bpo70+1] (1.5.8-3~bpo70+1 Debian Backports:/wheezy-backports [amd64])
Inst libxapian22 [1.2.12-2] (1.2.16-2~bpo70+1 Debian Backports:/wheezy-backports [amd64])
Conf libgpg-error0 (1.12-0.2~bpo70+1 Debian Backports:/wheezy-backports [amd64])
Conf libp11-kit0 (0.20.7-1~bpo70+1 Debian Backports:/wheezy-backports [amd64])
Conf augeas-lenses (1.2.0-0.2~bpo70+2 Debian Backports:/wheezy-backports [all])
Conf libaugeas0 (1.2.0-0.2~bpo70+2 Debian Backports:/wheezy-backports [amd64])
Conf libgeoip1 (1.6.2-4~bpo70+1 Debian Backports:/wheezy-backports [amd64])
Conf tmux (1.9-6~bpo70+1 Debian Backports:/wheezy-backports [amd64])
Conf dmidecode (2.12-2~bpo70+1 Debian Backports:/wheezy-backports [amd64])
Conf git-man (1:1.9.1-1~bpo70+2 Debian Backports:/wheezy-backports [all])
Conf git (1:1.9.1-1~bpo70+2 Debian Backports:/wheezy-backports [amd64])
Conf git-core (1:1.9.1-1~bpo70+2 Debian Backports:/wheezy-backports [all])
Conf haproxy (1.5.8-3~bpo70+1 Debian Backports:/wheezy-backports [amd64])
Conf libxapian22 (1.2.16-2~bpo70+1 Debian Backports:/wheezy-backports [amd64])
Reading package lists... Error!
E: Problem renaming the file /var/cache/apt/pkgcache.bin.9Ibnqk to /var/cache/apt/pkgcache.bin - rename (2: No such file or directory)
W: You may want to run apt-get update to correct these problems
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages will be upgraded:
 haproxy
1 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Inst haproxy [1.5.8-2~bpo70+1] (1.5.8-3~bpo70+1 Debian Backports:/wheezy-backports [amd64])
Conf haproxy (1.5.8-3~bpo70+1 Debian Backports:/wheezy-backports [amd64])

Control: severity -1 normal
Control: merge -1 717679

No*. At least if you are careful you have a few chances of noticing it.
The -V option e.g. tells you the versions. The download progress tells
you were stuff comes from.

Every version ever in existence in the last 17 years is effected.
That it survived 17 years is reason enough to not give it RC severity.

Also note that you need root-rights to run 'apt-get update' (or
equivalent) commands, much like you need root-rights to run 'rm -rf /'.
rm got over the years safeguards to prevent this, but this will never be
perfect and we are much in the same position. In the end root is the
only account who can do everything, but it should be asked if that means
that it should do everything. Maybe running multiple apt commands in
parallel is just in general not a good idea… and btw, neither it is for
dpkg or any other package management tool. Many things are heavily
interlocked here working in concert to make package installation
possible. Sometimes I think 'we' higherlevel parts like apt just made it
"too easy". On every other plattform which can be updated nobody would
ever ask for running stuff in parallel, simply because those happen in
total lockdown…

This isn't really about the Packages files, but about the Release files
and only on a second step about all the other files apt downloads. What
sounds like a trivial implementation happily explodes into a code
nightmare with only slightly less edgecases than decimal places of pi.
We are slowly getting better, but there is only so much you can do with
the resources we have…

Everyone using libapt is "effected" if you will, so even stuff like the
typical software-center. And not all these are started and/or always
working as root, so just 'locking' is not an option if you don't happen
to forbid every use of libapt as non-root in the process and only allow
libapt to be loaded by only one root application at the time. That would
immensly cripple the useability for next to no gain…


Best regards

David Kalnischkies