look's behavior became really unpredictable when it migrated from
bsdmainutils to util-linux.

$ look achi
Achille
Achille's
Achillean
Achilles
Achilles's
Achitophel
$ grep -ci ^achi /usr/share/dict/words
36
$

This is just one example but sometimes it works and sometimes it
doesn't.  It tends to behave consistently for a given prefix, but if
it chooses capitalized words or not varies (locale makes no difference
that I can see).  eg:

$ look cra | wc -l
518
$ grep -ci ^cra /usr/share/dict/words
609
$ diff -u <(grep -i ^cra /usr/share/dict/words) <(look cra) | head
--- /dev/fd/63  2020-10-31 08:00:51.587099984 +0000
+++ /dev/fd/62  2020-10-31 08:00:51.587099984 +0000
@@ -1,94 +1,3 @@
-Crabbe
-Crabbe's
-Craborchard
-Craborchard's
-Crabtree
-Crabtree's
-Cracker

Hello Jamie Heilman,

* Jamie Heilman <jamie@audible.transient.net> [201031 09:33]:

Thank you for your report.
[6 results]
$ grep -ci '^achi' /usr/share/dict/words
16

Anyway, util-linux's look uses a binary search and has requirements
on the input file. This is documented in the manpage look(1):

| As look performs a binary search, the lines in file must be
| sorted (where sort(1) was given the same options -d and/or -f
| that look is invoked with)

/usr/share/dict/words (in my case /usr/share/dict/american-english)
does not follow these requirements.

If you read the old look(1) manpage, it says under COMPATIBILITY:
| look uses a linear search by default instead of a binary search,
| which is what most other implementations use by default.

Now we are back at "what most other implementations use", so one
less compatibility issue.

If you would like to see improvements over this, please talk to
upstream about it.  Their contacts are:
      E-MAIL: util-linux@vger.kernel.org
      Web:    https://github.com/karelzak/util-linux/issues


Best wishes,
Chris

Hello,

The issue is that currently "look" is broken on debian/bullseye. I don't know
whether this package needs to be updated or the dictionary package(s).

Attached is a patch that removes the default -d and -f when no dictionary file
is given, which seems to mitigate this issue.

Regards,
    Yair.

Currently `look` is broken, according to upstream and Ubuntu bugs:
https://github.com/util-linux/util-linux/issues/1493[1]
https://bugs.launchpad.net/ubuntu/+source/util-linux/+bug/1971425[2]

aptitude show wamerican bsdextrautils

Package: wamerican
Version: 2020.12.07-2
State: installed
...
This package provides the file /usr/share/dict/american-english containing a list of English
words with American spellings. This list can be used by spelling checkers, and by programs
such as look(1).

Package: bsdextrautils
Version: 2.38-4
New: yes
State: installed
Automatically installed: yes
...

`bsdextrautils` cannot be not installed because it's in the dependencies of `man-db`.

Thanks,
Chris

--------
[1] https://github.com/util-linux/util-linux/issues/1493
[2] https://bugs.launchpad.net/ubuntu/+source/util-linux/+bug/1971425

If the sort order of the wordlist is changed (as #1040126 suggests), then that
would resolve this bug.

There would still be a question of whether the look tool ought to require the
wordlist to be sorted.  Other implementations of look(1) do (in absence of
explicit filename), so my feeling is that providers of /usr/share/dict/words
should ensure that the file is sorted with -d -f.  That being the case, my
feeling is that look(1) should indeed make use of that sort order: i.e. I feel
that the benefit of the reduced number of reads should outweigh the chance
of a misconfigured system (though see next para).

However, there is a question of how bsdextrautils should allow for the fact
that the existing version of scowl does supply an inappropriately ordered
wordlist: perhaps bsdextrautils should Conflict with existing versions, or
perhaps it should for now switch to not requiring sortedness, and later go back
to requiring orderedness and adding a Conflicts on existing scowl versions.

pjrm.

I'm just writing to confirm that after applying the attached patch (adding -f
to the sole occurrence of `sort' in debian/rules) to scowl-2020.12.07-2, the
test I gave before:

  while read -r i; do
    look "$i" | fgrep -q -w "$i" || echo " $i";
  done < /usr/share/dict/words

no longer identifies any words that look doesn't find (when /usr/share/dict/words
points via /etc/dictionaries-common/words to /usr/share/dict/american-english,
and look is provided by bsdextrautils 2.36.1-8+deb11u1, whose source package is
util-linux).

I do wonder whether there are any locales with interesting LC_COLLATE (or
LC_CTYPE) behaviour that could cause the above test to fail for some word
lists; though I'd think that that would be a bug to be fixed in util-linux, not
in scowl.  scowl's sort command in question already explicitly sets LC_ALL to C,
which seems like the right behaviour for scowl to me: I don't think one could
really do better even if the sort were delayed until installation time.

util-linux misc-utils/look.c calls setlocale(LC_ALL, "") at startup (i.e.
setting all locale categories from the environment), with no other occurrence
of setlocale in the source file; and its compare() function uses isalnum and
isblank (which are documented to use current locale), and strncasecmp (whose
behaviour is explicitly unspecified in its documentation, outside of POSIX
locale).  So it does seem like one could at least construct locales for which
this implementation of look would fail.  However, I have not tested this
conclusion (which is formed from a cursory look at the source by someone not
an expert in locales).

pjrm.