#807948 apt: 'update' fails with 'Couldn't create tempfiles for splitting up' InRelease files

Package:
apt
Source:
apt
Description:
commandline package manager
Submitter:
The Wanderer
Date:
2015-12-15 15:27:04 UTC
Severity:
important
#807948#5
Date:
2015-12-14 16:42:34 UTC
From:
To:
Dear Maintainer,

When I upgrade to apt 1.1.3 or later (tested with both that and 1.1.4),
and then run 'apt-get update' or 'apt update', I get results like the
following:

========
Ign:1 http://ftp.us.debian.org/debian stable InRelease
Hit:2 http://ftp.us.debian.org/debian testing InRelease
Err:2 http://ftp.us.debian.org/debian testing InRelease

Couldn't create tempfiles for splitting up
/var/lib/apt/lists/ftp.us.debian.org_debian_dists_testing_InRelease
  Could not execute 'apt-key' to verify signature (is gnupg installed?)
Hit:3 http://ftp.us.debian.org/debian stable Release

Err:4 http://ftp.us.debian.org/debian stable Release.gpg

  At least one invalid signature was encountered.
Ign:5 http://download.videolan.org/pub/debian/stable  InRelease

Hit:6 http://download.videolan.org/pub/debian/stable  Release
Err:7 http://download.videolan.org/pub/debian/stable  Release.gpg
  At least one invalid signature was encountered.
Hit:8 http://security.debian.org stable/updates InRelease
Err:8 http://security.debian.org stable/updates InRelease't create
tempfiles for splitting up
/var/lib/apt/lists/security.debian.org_dists_stable_updates_InRelease
  Could not execute 'apt-key' to verify signature (is gnupg installed?)
Hit:9 http://security.debian.org testing/updates InRelease
Err:9 http://security.debian.org testing/updates InRelease splitting up
/var/lib/apt/lists/security.debian.org_dists_testing_updates_InRelease
  Could not execute 'apt-key' to verify signature (is gnupg installed?)
Reading package lists... Done
W: An error occurred during the signature verification. The repository
is not updated and the previous index files will be used. GPG error:
http://ftp.us.debian.org/debian testing InRelease: Could not execute
'apt-key' to verify signature (is gnupg installed?)
W: An error occurred during the signature verification. The repository
is not updated and the previous index files will be used. GPG error:
http://ftp.us.debian.org/debian stable Release: At least one invalid
signature was encountered.
W: An error occurred during the signature verification. The repository
is not updated and the previous index files will be used. GPG error:
http://download.videolan.org/pub/debian/stable  Release: At least one
invalid signature was encountered.
W: An error occurred during the signature verification. The repository
is not updated and the previous index files will be used. GPG error:
http://security.debian.org stable/updates InRelease: Could not execute
'apt-key' to verify signature (is gnupg installed?)
W: An error occurred during the signature verification. The repository
is not updated and the previous index files will be used. GPG error:
http://security.debian.org testing/updates InRelease: Could not execute
'apt-key' to verify signature (is gnupg installed?)
W: Failed to fetch
http://ftp.us.debian.org/debian/dists/testing/InRelease  Could not
execute 'apt-key' to verify signature (is gnupg installed?)
W: Failed to fetch
http://security.debian.org/dists/stable/updates/InRelease  Could not
execute 'apt-key' to verify signature (is gnupg installed?)
W: Failed to fetch
http://security.debian.org/dists/testing/updates/InRelease  Could not
execute 'apt-key' to verify signature (is gnupg installed?)
W: Failed to fetch
http://ftp.us.debian.org/debian/dists/stable/Release.gpg  At least one
invalid signature was encountered.
W: Failed to fetch
http://download.videolan.org/pub/debian/stable/Release.gpg  At least one
invalid signature was encountered.
W: Some index files failed to download. They have been ignored, or old
ones used instead.
========

As the below should indicate, I do have gnupg installed.

Downgrading back to apt 1.0.10.2 (which is a manual process involving
multiple passes with dpkg, due to dependency problems) restores
functionality.

A problem which looks like the same issue has been reported by an Ubuntu
user:
https://bugs.launchpad.net/ubuntu/+source/apt/+bug/1524196

The configuration information below was captured from a system where the
downgrade back to 1.0.10.2 had been completed. The pins against apt and
aptitude-common are to prevent dist-upgrades from bringing back the
broken configuration again.

As far as I'm aware, nothing about my system should prevent this from
working. However, I've been unable to figure out how to get meaningful
information about what is failing.

If there is anything I can do to help track this down, please do not
hesitate to let me know.

#807948#10
Date:
2015-12-14 17:27:50 UTC
From:
To:
found 807948 1.1.3
notfound 807948 1.0.10.2
thanks

My mistake - I forgot to tweak the bug-report template's automatic
version info before sending it in. This is found in 1.1.3 and later, not
in 1.0.10.2.

This is the first time I've used the BTS commands, so I hope I've got
this right... I tried to correct it with the 'bts' command-line program
first, and saw no errors, but nothing seems to have happened to the bug
itself.

If this mail doesn't fix it, I'll leave things alone until someone can
look at this, or until I can get advice through other channels.

#807948#19
Date:
2015-12-14 17:47:03 UTC
From:
To:
(fixing up metadata to reflect report)
probably really about the tempfiles rather than this code itself as it
didn't change much.

I guess its permission related as this code is now executed as _apt
rather than as root. You can disable privilege dropping temporarily with
-o APT::Sandbox::User=root

Interesting might be the permissions of your $TMPDIR: stat /tmp $TMPDIR
Potentially also how you mount it (if you do it): mount | grep tmpfs
[I see that you haven't followed the systemd switch yet, which suggests
you could have other "new" default options not as default like a non-
persistent /tmp as well]

You could also try moving /var/lib/apt/lists away. There is no 'partial'
in this file path, so maybe some bad file is stuck in there doing bad
things. While at it: Anything interesting in the partial/ subdirectory
and does the mention file looks at least reasonable? The files are all
Hit's – what modification time have the files in the lists/ dir?

I presume 1.1.3 was the first apt you tried of the 1.1 series, right?

(That is a catch-all message printed after we had a failure higher up
the chain [which was probably very technical like this one] – with the
"most likely" cause added in a few layman terms as we do it in a few
other error messages as well.)


Best regards

David Kalnischkies

#807948#24
Date:
2015-12-15 04:31:05 UTC
From:
To:
Looks like my attempt got there a bit earlier, but thanks anyway, since
I wasn't sure I'd gotten it right.

(Is the use of -1 to represent "current bug" or suchlike documented
anywhere? I remembered seeing it used, but I couldn't find it in
https://wiki.debian.org/HowtoUseBTS,
https://www.debian.org/Bugs/server-control, or 'man bts', so I didn't
risk trying it.)

Agreed, that sounds likely - and thanks for the tip.
  File: ‘/tmp’
  Size: 53248           Blocks: 112        IO Block: 4096   directory
Device: fd02h/64770d    Inode: 2           Links: 73
Access: (0775/drwxrwxr-x)  Uid: ( 1000/wanderer)   Gid: ( 1004/  tmpdir)
Access: 2013-08-30 09:47:13.901246241 -0400
Modify: 2015-12-14 23:14:47.519608765 -0500
Change: 2015-12-14 23:14:47.519608765 -0500
 Birth: -

IOW, it's not world-writable, and the group involved is probably not a
standard one.

By comparison, on a different machine which doesn't have the problem,
/tmp is drwxrwxrwx root:root.

Now that I've looked at this, I vaguely recall that I set /tmp up this
way intentionally, not long after I built this system - but I can't
remember why. I think it was simply that I couldn't write to /tmp as my
normal user otherwise, but that doesn't seem to hold with the other
system; there may also have been a bit of thinking that the way /tmp
looked in 'ls / --color=auto' with drwxrwxrwx root:root was ugly, but if
so I haven't noticed it on the other system in the past few years.

This is probably the culprit. I'll investigate changing this in the
morning, when I have more time and less of a headache.

It might be worth trying to detect /tmp or $TMPDIR writability at some
point in the process, but I entirely understand if it's considered not
worth the code and/or the hassle.

This returns a bunch of things, starting with udev and including several
things under /run and one under /sys, but no /tmp or similar.

As it happens, /tmp is currently persistent on this computer, but is
non-persistent on the one which doesn't have the problem. That's got
nothing to do with the systemd switch AFAIK, however, and I'm pretty
sure it's orthogonal to the current problem.

I don't have this information ready to hand, since I always run the
update again after downgrading back to the working version, and that's
going to update the timestamps and suchlike. If it's important, which I
now think it probably isn't, I can do the dance to get it tomorrow or so.

It's the first to make it to testing, so yes.

I rather expected this would be the case. I mentioned gnupg just to
confirm that, yes, I had investigated the cause suggested in the error
message and it didn't seem to apply.

#807948#29
Date:
2015-12-15 15:22:56 UTC
From:
To:
I've changed this to match the described configuration of the machine
which works, and the error has disappeared.

(I first tried just adding '_apt' to the group involved, which has write
access to that directory, but that didn't work; I'm not really sure why.)

Unless it's worth trying to detect this type of case in the code and
either adapt to work around it or provide a more useful error message,
about the only thing I think might be called for would be update
documentation to at least provide a hint of where to look. (Someone
should probably also point this out to the person who reported the
Ubuntu bug I linked to; I may, if no one else does.)

If that's not worth it either, this bug can probably be closed as (a
somewhat unusual form of) operator error.