man-db has had dpkg triggers for almost four years.  These triggers make
the cron jobs entirely redundant.  Please consider removing these cron
jobs.

Thanks,
Josh Triplett

My main concern here is that I have no idea how many people are relying
on apropos output for locally-installed (i.e. non-packaged) manual
pages; removing the cron jobs would break that unless people knew to run
mandb by hand.  I rather suspect that the probability of that is higher
outside the population of Debian developers ...

but that doesn't mean people don't use it.  (I more frequently tend to
find myself using zgrep over the entire contents of all manpages, rather
than just doing a keyword search over a subset of those contents.)  I'd
personally prefer to just turn it off entirely, much like the locate
database.  (How crazy would you consider it to split the apropos
functionality into a separate package, much like locate split off of
findutils?)

However, the need to run mandb for new manpages in /usr/local seems no
different than the need to run ldconfig for new libraries in /usr/local
(other than the severity of what happens if you don't).  I think you
could reasonably convey that via NEWS.Debian.gz.

One other thought: what if you made mandb record the stat of the
directories in the manpath, and then had any tools relying on the mandb
check that stat and warn if mandb needs to run again?

- Josh Triplett

I see it used quite a bit, and not just by me. :-)

Nowadays there's 'man -K' too for that, although that doesn't rely on
the database.  It should be very significantly faster than zgrep.
mainly that mandb is slow?  If so, I have microbenchmarks that strongly
indicate that the slowness isn't intrinsic, and if I ever manage to put
enough spare hours in a row I intend to fix that.  It really shouldn't
need to take more than maybe a few seconds even on not terribly exciting
hardware, and certainly nowhere near as much as locate.  See #630799.

My basic concern here is that splitting packages is forever, or very
nearly, and so I'm reluctant to do so to work around fixable performance
problems.

It's better for software to just work rather than to require
documentation explaining how to make it work, though.

That might be workable, I guess, although I worry that it would be
annoying for users without administrative privileges.  (apropos used to
attempt to raise privileges and update the database automatically; that
was a disaster area and removing it was a big improvement.)

Cheers,

Handy, but in general I find man's behavior when multiple manpages match
extremely suboptimal.  It immediately opens the first matching page, and
forces me to walk through each page one-by-one without the ability to
backtrack.  Grep shows me pagefuls of search results at a time, and I
can skim and pick what I need.

That doesn't relate to the bug report at hand, though.

No, mandb's speed doesn't concern me here.  I reported this issue
because in general I consider it suboptimal whenever something runs on a
periodic basis when it really only needs to run when things change.
Very few cron jobs really need to run periodically; most of them
represent workarounds for the inability to detect changes and run when
those changes occur.

Looking at cron.daily, cron.weekly, and cron.monthly on my system, every
single cronjob would work better in some other form that didn't blindly
run on a periodic basis.

I don't consider this just a performance problem; I'd like to avoid
using periodic cronjobs as a substitute for "run when needed".

I never expect software in /usr/local to Just Work; I have Debian
packages when I want things to Just Work. :)

/usr/local already represents an area that requires manual maintenance
by the administrator: ldconfig, mandb, compiling emacs and Python
scripts, running fc-cache for new fonts, running whatever magic thing
TeX needs to update its indexes.  Cronjobs seem highly suboptimal for
these kinds of updates: they run periodically whether they have work to
do or not, they delay updates until well after the install has occurred,
and they only cover a single package's needs.

Perhaps in the future it might make sense to have some standardized
mechanism to let packages update on changes to /usr/local.  For
instance, packages could set file triggers on appropriate parts of
/usr/local, and administrators could have an easy way to run all
appropriate dpkg triggers after doing a "make install" or equivalent.

An increasingly rare class of users. :)

But given that fixing it just requires running "mandb", it seems like an
extremely simple support issue for an administrator to fix, and add to
their checklist of "things to do after 'make install'".

Yeah, no good can come of that approach.

(That said, in the future when we have some sensible service supervision
mechanism, apropos could poke some system daemon and have it fire off a
mandb, without attempting to raise its own privileges.  Alternatively,
mandb could automatically get triggered shortly after any change to
directories on the manpath.)

- Josh Triplett

Add the -w option.

We may be at an impasse here, because without being convinced that use
of local manpaths is truly falling off and that nobody cares about
running apropos in them any more (and the bug reports I occasionally get
gently suggest otherwise, although man-db gets a low enough volume of
bugs that it's hard to ascribe statistical significance to anything),
I'm not sure I'm prepared to make a change that I can well believe some
users would consider a regression.

I guess I could make it configurable somehow, but of course they're
conffiles so technically they already are.  This seems to be a debate
about defaults.

They're suboptimal, yes; but they're the only stopgap that exists right
now.  And I don't in general agree that a situation being generally
terrible means that small steps to try to make it better aren't
worthwhile.

Yes, that would be nice.

I'll certainly consider making apropos whine about it.  As for the rest,
I'm not sure (and it seems fairer to say that straight out rather than
just stalling).

Some of this is very likely coloured by my experience a couple of jobs
back as an unprivileged user on a variety of different Unix systems
where they hadn't been correctly configured to build the apropos
database and so apropos was useless.  We had lots of locally installed
software for development - particularly on Solaris which came with
terrible userspace software by default - and so it was actually
annoying, but they were build systems more than anything else and nobody
really wanted to bother about that kind of detail.  It was always a
point of pride with me that man-db had multiple ways to ensure that it
didn't have this problem.  (Yes, slightly obsessive, but you'd kind of
hope that the man-db maintainer was obsessive about manual pages.)

Now, I'm fairly sure that Debian-based systems are less prone to that
kind of thing.  I certainly hope so.  On the other hand, I live in a
bubble where this is concerned; I've been working full-time on Ubuntu
since 2004, and obviously we're not exactly big on making extensive use
of locally installed software for our own purposes, so I like to work on
the general assumption that I use a lower percentage of unpackaged
software than virtually everyone else.

If it were a daemon it could use inotify watches; but I definitely don't
want to go to the effort of converting it to a daemon.  A sufficiently
capable service supervision framework could perhaps install inotify
watches for it, but I would be concerned about that triggering many
times per update; and given that the performance problem with mandb is
chiefly "too many forks", that's something I'd prefer to avoid.

Cheers,

Nice!  Thanks.

Sure, once you start providing any functionality (optimal or otherwise),
changing it can potentially feel like a regression.

I don't mean to suggest that nobody uses manpages from /usr/local, or
that nobody uses apropos on those pages.  I just mean that the current
cronjob seems like a suboptimal solution, particularly for the large
number of people who *don't* have anything in /usr/local, but even for
those who do.

Yeah, I don't see the need for any configurability beyond the current
option of "rm /etc/cron.*/man-db".  Definitely a debate about defaults.

True.  But I'd like to look for the right general-purpose solution, and
try to apply that across many packages.  And one potentially right
general-purpose solution, for one sense of "right", might be "document
what an administrator needs to do when installing local software".

Filed as bug 667438.

I appreciate that.

One interesting idea: given that disks have gotten much faster, and that
most people don't have a huge number of local manpages installed, what
about tracking the directory timestamps (and manpage timestamps) as
before, making -k use the mandb database when up to date, and falling
back to searching the individual manpages like -K when not up to date?
Then, the complaint from apropos would just amount to "please have your
administrator do $foo so this runs faster", rather than "this won't work
at all until your administrator does $foo, but you might try -K".

I certainly wouldn't suggest making man-db have its own daemon; I meant
more that a sufficiently capable service supervisor could do inotify
itself.  And I agree that triggering it each time makes no sense; the
ideal use of inotify watches would look something like "wait for $TIME
to pass after the last inotify trigger, then run it".

- Josh Triplett