Hi,

with the last upgrade of bash-completion completing hostnames has become
slow (takes more than two seconds). My /etc/ssh/ssh_known_hosts file has
quite some size (23084 lines / 12 MB), but this was not a problem with the previous
implementation.

I guess the problem is the while read … done < known_hosts_file loop, where
the shell has to read() one byte at a time, causing a massive amount of
syscalls (which have become even more costly this year).


Regards,
Timo

Hi, thanks for the report.

OK, sounds like a lot of time.
to fix bug #848125 [2].  Running the following command against current
upstream sources shows that there are no newer patches related to this
(lines 1479 to 1665 contain the function, _known_hosts_real, that
parses the known_hosts file):

  git log --stat -L1479,1665:bash_completion
  (lastest commit is the patch that I backported [1])

I think we need to forward this problem report upstream.  Do you have
more information that I could add to it (maybe the syscall trace you
ran)?

PS: I don't want to remove the patch from Debian, because it would only
solve the problem you are reporting until bash-completion 2.9 is
released.

[1] https://github.com/scop/bash-completion/commit/71ac42b98c0eba39819fb8478d6443032c6ef6f1
[2] https://bugs.debian.org/848125

Hallo Gabriel F. T. Gomes,

25.11.18 15:35 Gabriel F. T. Gomes:

OK, it is not the one-byte-per-read()-problem, which still occurs with pipes;
bash has an optimisation for seek()able files and read()s 128 bytes per call,
then seek()s back to behind the line terminator.

Before reporting the bug I read the changelog and a quick research found that
code change as a debian patch.

$ time bash -c 'while read -ra line; do :; done' < kh_file

real    0m0,811s
user    0m0,749s
sys     0m0,059s

I can't provide my /etc/ssh/ssh_known_hosts file because it contains
information some people consider confidential. But I can say that for a host
in that file there are several names (up to 221) and multiple key( type)s aka
duplicate hostlists (up to 3).

Of course.

I'll try to hack something with the same features using only essential tools
(cut, grep, tr, sed, …) that will do in under 0,1s with the same input.


Grüße
Timo