Steps to reproduce:
- Install debian using recently published debian-bookworm-DI-
rc1-amd64-netinst.iso
- At first boot as a very first thing to do type "apt-get install a<Tab>" (as
root, in normal root login shell)
- You will see that bash-completion doesn't work
- If you type "apt-get install apt<Enter>", and try "apt-get install a<Tab>"
again, you will see that bash-completion works

This situation is very bad. Imagine you installed Debian. Of course, the first
thing you may try to do is install some package to it. So it is quite possible
you will type "apt-get install ..." as a first thing. And you will see that
bash-completion doesn't work!

(Don't confuse "apt-get install a<Tab>" with "apt-get insta<Tab>")

I think I know why this happens. d-i has file /usr/lib/finish-
install.d/60cleanup. This file calls "apt-get clean". "apt-get clean" removes
"/var/cache/apt/*.bin". And "/var/cache/apt/*.bin" are needed for bash-
completion to work with apt.

If I remove /usr/lib/finish-install.d/60cleanup, then this bug disappears.

So, if you want to quickly fix this bug, simply delete /usr/lib/finish-
install.d/60cleanup.

I don't know what should really be done. I can list possible solutions:

- Fix d-i. Remove /usr/lib/finish-install.d/60cleanup or replace it with
something else
- Fix "apt-get clean". Make it not remove /var/cache/apt/*.bin
- Fix "bash-completion". Make it work even if /var/cache/apt/*.bin are not
present (for example, by recreating them on-the-fly)

(Disregard "System information" below)

Hi Askar,

and thanks for the report.

Askar Safin <safinaskar@mail.ru> (2023-04-20):

d-i is merely exposing a bug in bash-completion, so I don't think we
should change anything there.

Clearly one of these. While I'm no apt maintainer and can't speak for
the semantics behind “apt-get clean”, it seems to me bash-completion's
relying on files that might not be there is the crux of the problem.


Cheers,

Четверг, 20 апреля 2023, 23:33 +03:00 от "Cyril Brulebois" <kibi@debian.org>:
It seems apt completion is located in /usr/share/bash-completion/completions/apt, which belongs to apt package.
So, anyway this belongs to apt package, so I reassigned


==
Askar Safin

Hi,

before I am playing bug ping-pong (as I don't think we can do much about
this on the apt side), let me ask a d-i question first:

Is it perhaps just wanting to remove /var/cache/apt/archives/*.deb ?
If so, could you switch to: -o APT::Keep-Downloaded-Packages=false
(well, probably in a conf file rather than on each command line).

This instructs apt(-get) to remove the deb file right after it has
installed them with dpkg, which might even be beneficial for space
constraint installation targets. At the very least it is a much used
& tested option as it is the default with apt (but not apt-get).


d-i could also "just" run 'apt-cache gencaches' to create the caches;
but perhaps it is intended that the caches are gone (docker people e.g.
like to do that I am told, but they nuke other things as well…).

The completion script explicitly disables the recreation on-the-fly as
creating the files takes a while robbing users for many seconds of their
interactivity. So, we can't just "fix" the completion script as that has
a(nother?) set of users complain as well.


Best regards

David Kalnischkies

Hi David,

David Kalnischkies <david@kalnischkies.de> (2023-05-04):

Entirely fair, thanks. Putting debian-boot@ in copy since currently the
bug is assigned to apt… and others might have some feedback about my
proposal(s).

Checking occurrences of `apt-get clean` (I didn't check for variations)
in our packages, I see two callers.

One is pkgsel (which deals with tasksel and pulls many/most packages),
and one later on in finish-install (which as you can guess runs at the
end).

There are various apt-get (not clean) calls, some of them wrapped, and I
have no idea what the consequences of tweaking them could be. In any
case, while deplying a conf file during the installation process could
be attempted, it's far too late to try toying with that…

We can probably keep track of the suggestion via some bug report against
debian-installer, so that someone can investigate that post-Bookworm
(probably removing the possible tweak discussed below).

OK, thanks.
https://bugs.debian.org/586434

With all the backstory, the interactions/dependencies between
bash-completion and apt's internal files became clear, thanks.

For Bookworm, I'm tempted to add `apt-cache gencaches` after `apt-get
clean` in finish-install, restoring the files bash-completion relies on.

That'd waste some little time, rebuilding some of the files that were
just deleted… but at least we wouldn't risk anything in the rest of the
installation process.

(Maybe making the extra call conditional to bash-completion's being
installed? Not sure about the “small optimization” vs. “different apt
files depending on the installed packages” balance.)


Cheers,

Current apt behavior also breaks official debian docker images, I
reported this here:
https://github.com/debuerreotype/debuerreotype/pull/153 . The images'
developer said this is apt completion scripts' problem

To be clear, there was nothing _certain_ in my response, merely that
my first impression matched Cyril's for the very similar issue on d-i.

I'd be very happy to spend some time with APT folks to figure out a
better configuration for the Docker images (both specifically on this
issue and in general!), especially as there seems to be some tension
around our current configuration that was unintentional and I'm sorry
for. 🙇

(Having APT maintainers say "you should do X to accomplish what you're
looking for" is a much more defensible change than anything I could
possibly make up alone! 💕)

♥,
- Tianon
  4096R / B42F 6819 007F 00F8 8E36  4FD4 036A 9C25 BF35 7DD4

David Kalnischkies:

Deleting /var/cache/apt/*.bin and then running "apt-cache gencaches"
takes 0.22 on my (loaded) system.

I propose regenerate caches in completion scripts only if they are not
present. This will change situation from "not working at all" (in
default d-i install and in official docker image) to "working slowly".
Also, "working slowly" is just minor annoyance. But "not working"
means that the user doesn't know how to fix. Yes, merely calling
"apt-get install apt" will regenerate caches (in d-i case, not in
docker case), but how the user should know this? The solution is not
discoverable

Okay, sorry for misunderstanding.

Hi again,

Cyril Brulebois <kibi@debian.org> (2023-05-04):

[…]

No feedback in either direction at the moment, I'm going ahead as the
Bookworm clock is ticking.

Doing so via the control commands above.

I'll do that in finish-install and add a pointer to this bug report; I
won't add a Closes statement though, as I'm not sure what other users
will be implementing (for Docker etc.)… I'll leave it up to you to
decide what's best regarding this bug report (the -1 one).


Cheers,

I did some experiments and wrote a patch!

Okay, so, I spend some more time thinking about this. And I think the
bug is in apt's completion scripts. Because that *.bin files are
simply cache. And lack of caches should not break program! This
follows from my interpretation of FHS 3.0 5.5 (
https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch05s05.html ). And
that *.bin files are located in /var/cache. And debian policy refers
to FHS 3.0 ( https://www.debian.org/doc/debian-policy/ch-opersys.html#file-system-hierarchy
). So, the bug is in apt's completion scripts and we should simply
remove "--no-generate" from them. And hence I attach a patch, which
does exactly that.

This will cause completion scripts to regenerate caches as needed.
This will work even if *.bin files are not present. And this will work
even if we are in unprivileged bash, I tested this.

Okay, so you can ask: what are speed implications of this? Well, I
tested this on my main system (Intel Core i7) and I feel that even
full cache regeneration takes zero time, i. e. it is absolutely
instant.

So I started qemu-system-x86_64 without -enable-kvm and tested there.

Results are this: if caches are not present, then bash-completion is
slow, it takes something like 10 seconds (in qemu in emulation
mode!!!). But if caches are present, then bash-completion feels fast.
So, it seems that apt is smart, it regenerates cache only if needed.
So, there is no need to insert special "if" in completion scripts to
check whether we should regenerate caches, apt already does this.

Also, I found that file /usr/share/bash-completion/completions/apt is
owned by apt, but file /usr/share/bash-completion/completions/apt-get
is owned by bash-completion . So, I apply patch to "apt" only. Needed
patch for "apt-get" is essentially the same.

Also, I think "apt-get" (and similar) scripts should be transferred
from "bash-completion" package to "apt".

This patch is for current apt's "main",
9da15d149f97e0f26cf5b7e32405512a5e63523c .

diff --git a/completions/bash/apt b/completions/bash/apt
index 59a8eaa94..ff3d7bd24 100644
--- a/completions/bash/apt
+++ b/completions/bash/apt
@@ -173,6 +173,7 @@ _apt()
     fi

     # specific command arguments
+    # we don't pass "--no-generate" to "apt-cache" because we want
completion to work even if caches are not present (#1034650)
     if [[ -v command ]]; then
         case $command in
             remove|purge|autoremove|autopurge)
@@ -187,7 +188,7 @@ _apt()
                 return 0
                 ;;
             show|list|download|changelog|depends|rdepends)
-                COMPREPLY=( $( apt-cache --no-generate pkgnames "$cur" \
+                COMPREPLY=( $( apt-cache pkgnames "$cur" \
                     2> /dev/null ) )
                 return 0
                 ;;
@@ -195,7 +196,7 @@ _apt()
                 if [[ "$cur" == .* || "$cur" == /* || "$cur" == ~* ]]; then
                     _filedir "deb"
                 else
-                    COMPREPLY=( $( apt-cache --no-generate pkgnames "$cur" \
+                    COMPREPLY=( $( apt-cache pkgnames "$cur" \
                         2> /dev/null ) )
                 fi
                 return 0
@@ -204,7 +205,7 @@ _apt()
                 if [[ "$command" == build-dep && ( "$cur" == .* ||
"$cur" == /* || "$cur" == ~* ) ]]; then
                     _filedir "dsc"
                 else
-                    COMPREPLY=( $( apt-cache --no-generate pkgnames "$cur" \
+                    COMPREPLY=( $( apt-cache pkgnames "$cur" \
                         2> /dev/null ) $( apt-cache dumpavail | \
                         command grep "^Source: $cur" | sort -u | cut
-f2 -d" " ) )
                 fi

please let me know if there are good options for me