#782501 "apt-get clean" deletes temporary files written by a running "apt-get update"

Package:
apt
Source:
apt
Description:
commandline package manager
Submitter:
Tim Landscheidt
Date:
2015-04-16 09:33:08 UTC
Severity:
normal
#782501#5
Date:
2015-04-13 09:54:33 UTC
From:
To:
Dear Maintainer,

unattended-upgrades regularly complained about
(cf. https://bugs.launchpad.net/ubuntu/+source/unattended-upgrades/+bug/1366708):

| E:Problem renaming the file /var/cache/apt/pkgcache.bin.WfLyE7 to /var/cache/apt/pkgcache.bin - rename (2: No such file or directory)
| W:You may want to run apt-get update to correct these problems

Googling brought up
https://bugs.launchpad.net/landscape-client/+bug/1091137:

| When the landscape-client runs apt-get update, it can return an error like this:

| E: Problem renaming the file /var/cache/apt/pkgcache.bin.PGPOhV to /var/cache/apt/pkgcache.bin - rename (2: No such file or directory)
| W: You may want to run apt-get update to correct these problems

| This can happen if someone is running apt-get clean, while the update is happening. apt-get clean
| removes all pkgcache.bin files, including the temporary ones that apt-get update uses
| to rebuild the cache. This error is harmless, since pkgcache.bin should also have been
| removed, and thus the next time the cache will be opened the cache will be rebuilt.

| I.e, we should ignore the error above as to not cause intermittent alerts in the Landscape
| server, that the user can't do anything to resolve.

And indeed the error is reproducable by:

| scfc@toolsbeta-jessie3:~$ while sudo apt-get clean; do :; done & sudo apt-get update; kill -HUP %%
| [1] 6241
| Hit http://apt.wikimedia.org jessie-wikimedia InRelease
| Hit http://mirrors.wikimedia.org jessie InRelease
| Hit http://mirrors.wikimedia.org jessie-updates InRelease
| Hit http://apt.wikimedia.org jessie-wikimedia/main Sources
| Hit http://apt.wikimedia.org jessie-wikimedia/backports Sources
| Get:1 http://mirrors.wikimedia.org jessie/main Sources/DiffIndex [7,876 B]
| Hit http://security.debian.org jessie/updates InRelease
| Hit http://apt.wikimedia.org jessie-wikimedia/thirdparty Sources
| Hit http://apt.wikimedia.org jessie-wikimedia/main amd64 Packages
| Hit http://apt.wikimedia.org jessie-wikimedia/backports amd64 Packages
| Hit http://apt.wikimedia.org jessie-wikimedia/thirdparty amd64 Packages
| Get:2 http://mirrors.wikimedia.org jessie/main amd64 Packages/DiffIndex [7,876 B]
| Get:3 http://mirrors.wikimedia.org jessie/main Translation-en/DiffIndex [7,876 B]
| Hit http://mirrors.wikimedia.org jessie-updates/main Sources
| Hit http://mirrors.wikimedia.org jessie-updates/main amd64 Packages
| Hit http://mirrors.wikimedia.org jessie-updates/main Translation-en
| Ign http://apt.wikimedia.org jessie-wikimedia/backports Translation-en_US
| Ign http://apt.wikimedia.org jessie-wikimedia/backports Translation-en
| Ign http://apt.wikimedia.org jessie-wikimedia/main Translation-en_US
| Ign http://apt.wikimedia.org jessie-wikimedia/main Translation-en
| Ign http://apt.wikimedia.org jessie-wikimedia/thirdparty Translation-en_US
| Ign http://apt.wikimedia.org jessie-wikimedia/thirdparty Translation-en
| Hit http://security.debian.org jessie/updates/main Sources
| Hit http://security.debian.org jessie/updates/main amd64 Packages
| Hit http://security.debian.org jessie/updates/main Translation-en
| Fetched 23.6 kB in 1s (12.2 kB/s)
| Problem renaming the file /var/cache/apt/srcpkgcache.bin.ZDdbP3 to /var/cache/apt/srcpkgcache.bin - rename (2: No such file or directory)
| Problem renaming the file /var/cache/apt/pkgcache.bin.93xAUQ to /var/cache/apt/pkgcache.bin - rename (2: No such file or directory)
| The package lists or status file could not be parsed or opened.
| Can't call method "policy" on an undefined value at /usr/bin/apt-show-versions line 56.
| E: Problem executing scripts APT::Update::Post-Invoke-Success 'test -x /usr/bin/apt-show-versions || exit 0 ; apt-show-versions -i'
| E: Sub-process returned an error code
| scfc@toolsbeta-jessie3:~$

IMHO apt-get should not delete temporary files owned by another
process.

#782501#10
Date:
2015-04-15 13:44:50 UTC
From:
To:
I wrote:

That may unnecessarily narrow down potential fixes for this
bug.  Of course it would be possible as well (and maybe
preferable) for "apt-get update" (and other commands?) to
create their temporary files elsewhere or, depending on com-
patibility and privilege considerations, use linkat(2) with
AT_EMPTY_PATH.

Tim

#782501#15
Date:
2015-04-15 13:54:36 UTC
From:
To:
Am 15.04.2015 15:48 schrieb "Tim Landscheidt" <tim@tim-landscheidt.de>:

Clean specifically includes the temporary files, so elsewhere is not an
option. Linkat would be nice in theory, but complicates the code even more.

Locking seems like the best idea IMO, not sure how things currently work
for cache though.

#782501#20
Date:
2015-04-16 09:28:28 UTC
From:
To:
"This flag is Linux-specific". If we fix that we shouldn't leave our
hurd and kfreebsd friends out in the cold if possible…
(Its friend open(2) O_TMPFILE is even linux and filesystem-specific)

The cachefiles are created by pkgCacheGenerator::MakeStatusCache in
apt-pkg/pkgcachegen.cc. Adding some write-locking here and trying to get
one in pkgCacheFile::RemoveCaches in apt-pkg/cachefile.cc and if not
skipping the file should be possible and enough to fix that, right?

So, Tim, you seem to be interested in seeing this fixed in the future,
may I interest you in writing a patch?  Feel free to ask me (or the
others) by mail or IRC (#debian-apt) anything (optionally even stuff
related to apt). ;)


Best regards

David Kalnischkies