#576810 perlindex: Perlindex cron job never finishes

#576810#5
Date:
2010-04-07 12:01:53 UTC
From:
To:
When started by cron, perlindex will run until killed - on one occasion it
ran for 11.5 hours - using ~90-95% CPU.

#576810#8
Date:
2010-04-07 12:41:28 UTC
From:
To:
-=| Joe Dennigan, Wed, Apr 07, 2010 at 01:01:53PM +0100 |=-

Maybe because of https://rt.cpan.org/Public/Bug/Display.html?id=20220

#576810#13
Date:
2010-04-11 12:48:25 UTC
From:
To:
tags 576810 moreinfo
thanks

-=| Joe, Thu, Apr 08, 2010 at 12:53:39AM +0100 |=-

The module actualy reading the files is Pod::Text. Before reassigning
to 'perl', I prefer to be sure that the bug is indeed there.

Can you reproduce the hang and see which file was processed at that
time? I have wine installed (as in the CPAN bugreport) but perlindex'
cron job never hung for me.

#576810#20
Date:
2010-05-17 19:01:35 UTC
From:
To:
today after the upgrade of Squeeze I have a perlindex process using 99%
of one CPU and never finish his job.

# lsof | grep perlindex
perlindex 25210        root  cwd       DIR                8,2      4096
          2 /
perlindex 25210        root  rtd       DIR                8,2      4096
          2 /
perlindex 25210        root  txt       REG                8,2    103048
   10330221 /bin/dash
perlindex 25210        root  mem       REG                8,2   1379752
    1073926 /lib/libc-2.10.2.so
perlindex 25210        root  mem       REG                8,2    123488
    1073902 /lib/ld-2.10.2.so
perlindex 25210        root    0r      CHR                1,3       0t0
       1429 /dev/null
perlindex 25210        root    1w      CHR                1,3       0t0
       1429 /dev/null
perlindex 25210        root    2w      CHR                1,3       0t0
       1429 /dev/null
perlindex 25210        root    3r      DIR                8,2      4096
          2 /
perlindex 25210        root   10r      REG                8,2       139
    1255755 /etc/cron.daily/perlindex
perlindex 25210        root   11w     FIFO                0,8       0t0
     186893 pipe
perlindex 25210        root   12w     FIFO                0,8       0t0
     186894 pipe
perlindex 25211        root  cwd       DIR                8,2      4096
    6972286 /usr/share/perl5
perlindex 25211        root  rtd       DIR                8,2      4096
          2 /
perlindex 25211        root  txt       REG                8,2      6928
    6974189 /usr/bin/perl
perlindex 25211        root  mem       REG                8,2     37728
    4005901 /usr/lib/perl/5.10.1/auto/List/Util/Util.so
perlindex 25211        root  mem       REG                8,2     12672
    7643774 /usr/lib/perl5/auto/Cwd/Cwd.so
perlindex 25211        root  mem       REG                8,2     19920
    4022275 /usr/lib/perl/5.10.1/auto/IO/IO.so
perlindex 25211        root  mem       REG                8,2   1501800
    6974345 /usr/lib/libdb-4.7.so
perlindex 25211        root  mem       REG                8,2     57936
    3014657 /usr/lib/perl/5.10.1/auto/DB_File/DB_File.so
perlindex 25211        root  mem       REG                8,2     18120
    3915777 /usr/lib/perl/5.10.1/auto/Fcntl/Fcntl.so
perlindex 25211        root  mem       REG                8,2     35104
    1073908 /lib/libcrypt-2.10.2.so
perlindex 25211        root  mem       REG                8,2   1379752
    1073926 /lib/libc-2.10.2.so
perlindex 25211        root  mem       REG                8,2    130841
    1073936 /lib/libpthread-2.10.2.so
perlindex 25211        root  mem       REG                8,2    530728
    1073934 /lib/libm-2.10.2.so
perlindex 25211        root  mem       REG                8,2     14696
    1073938 /lib/libdl-2.10.2.so
perlindex 25211        root  mem       REG                8,2   1491944
    3055622 /usr/lib/libperl.so.5.10.1
perlindex 25211        root  mem       REG                8,2    123488
    1073902 /lib/ld-2.10.2.so
perlindex 25211        root    0r      CHR                1,3       0t0
       1429 /dev/null
perlindex 25211        root    1w      CHR                1,3       0t0
       1429 /dev/null
perlindex 25211        root    2w      CHR                1,3       0t0
       1429 /dev/null
perlindex 25211        root    3r      DIR                8,2      4096
          2 /
perlindex 25211        root    4r      REG                8,2     20066
    2719997 /usr/bin/perlindex
perlindex 25211        root    5u      REG                8,2  13070336
    2899969 /usr/share/man/index_if
perlindex 25211        root    6u      REG                8,2   2633728
    2899970 /usr/share/man/index_idf
perlindex 25211        root    7u      REG                8,2    331776
    2899971 /usr/share/man/index_seen
perlindex 25211        root    8u      REG                8,2    344064
    2899972 /usr/share/man/index_fn
perlindex 25211        root    9r      REG                8,2       148
    1261789 /etc/perlindex/config


Today I have upgrade/installed the following packages:
# grep 2010-05-17 /var/log/dpkg.log | grep " installed" |cut -d ' ' -f 5
doc-base
hicolor-icon-theme
man-db
shared-mime-info
gnome-menus
desktop-file-utils
menu
c++-annotations-contrib
c++-annotations-html
gnome-games-extra-data
libcanberra0
libcanberra-gtk0
gnome-session-canberra
kdebase-runtime-data
kdegames-mahjongg-data
libcanberra-gtk-module
libieee1284-3
libnet-dbus-perl
libwww-perl
xbl
menu
man-db
gnome-menus
desktop-file-utils
hicolor-icon-theme
shared-mime-info
menu
libattica0
libkdecore5
libkdeui5
libsolid4
libkio5
libkparts4
libkpty4
libkde3support4
libkdesu5
libkdnssd4
libkfile4
libkjsapi4
libktexteditor4
libkhtml5
libkmediaplayer4
libknewstuff2-4
libknewstuff3-4
libknotifyconfig4
libkutils4
libnepomuk4
libnepomukquery4a
libssh-4
libkjsembed4
libkntlm4
libkrosscore4
libpolkit-qt-1-0
kdoctools
kdelibs-bin
kdelibs5-data
kdelibs5-plugins
shared-desktop-ontologies
libthreadweaver4
libplasma3
plasma-scriptengine-javascript
kdebase-runtime
libkdegames5
kblackbox
libkimproxy4
libkrossui4
kdelibs5
kgoldrunner
kmahjongg
konquest
kshisen
ksirk
ksudoku
libkcddb4
palapeli-data
palapeli
menu

Let me know if I can give you more information.

Ciao
Davide

#576810#25
Date:
2010-05-20 18:30:13 UTC
From:
To:
I make a boot each day on my Debian Desktop.
Only on 17/05/2010 I have such problem.

Probably installing/upgrading some package may cause problem to
perlindex and a new boot solve the problem.

This problem can be serious if you run a server that is never shout down
and perlindex never stop (it consume a lot of CPU, generate hot
temperature, ...).

I have three Squeeze machine and only in the 64bit one I have had that
problem.

On other Squeeze Desktop (i386) I don't have installed the followed
packages that are upgrade before the problem:
doc-base
c++-annotations-contrib
c++-annotations-html
gnome-session-canberra
kdebase-runtime-data
kdegames-mahjongg-data
xbl
libattica0
libkdecore5
libkdeui5
libsolid4
libkio5
libkparts4
libkpty4
libkde3support4
libkdesu5
libkdnssd4
libkfile4
libkjsapi4
libktexteditor4
libkhtml5
libkmediaplayer4
libknewstuff2-4
libknewstuff3-4
libknotifyconfig4
libkutils4
libnepomuk4
libnepomukquery4a
libssh-4
libkjsembed4
libkntlm4
libkrosscore4
libpolkit-qt-1-0
kdoctools
kdelibs-bin
kdelibs5-data
kdelibs5-plugins
shared-desktop-ontologies
libthreadweaver4
libplasma3
plasma-scriptengine-javascript
kdebase-runtime
libkdegames5
kblackbox
libkimproxy4
libkrossui4
kdelibs5
kgoldrunner
kmahjongg
konquest
kshisen
ksirk
ksudoku
libkcddb4
palapeli-data
palapeli

Ciao
Davide

#576810#30
Date:
2010-05-20 18:47:09 UTC
From:
To:
ouch!
I don't have installed perlindex on that machine :-(

sorry

Ciao
Davide

#576810#35
Date:
2010-06-05 10:25:05 UTC
From:
To:
I have had this problem again after an upgrade

$ grep 2010-06-05 /var/log/dpkg.log | grep " installed" |cut -d ' ' -f 5
libsepol1
man-db
doc-base
gnome-menus
desktop-file-utils
menu
postgresql-common
install-info
bogofilter-common
bogofilter-bdb
bogofilter
debian-reference-common
debian-reference-en
debian-reference
debian-reference-de
debian-reference-es
debian-reference-fr
debian-reference-it
debian-reference-ja
debian-reference-pl
debian-reference-pt-br
debian-reference-zh-cn
debian-reference-zh-tw
gir1.0-freedesktop
libgirepository1.0-0
gir1.0-glib-2.0
gnome-randr-applet
libcommons-logging-java
libtelepathy-glib0
myspell-en-us
openoffice.org-thesaurus-en-us
texlive-common
texlive-base
texlive-latex-base-doc
texlive-metapost-doc
libois-1.2.0
libois-dev
texlive-metapost-doc
texlive-latex-base-doc
texlive-base
tex-common
texlive-metapost
texlive-latex-base
texlive-luatex
menu
texlive-luatex
texlive-latex-base
tex-common


If I look at package upgraded with the previous problem I can see that
these are present in either:

desktop-file-utils
doc-base
gnome-menus
man-db
menu

I don't know if this can help

Ciao
Davide

#576810#40
Date:
2011-11-04 15:00:59 UTC
From:
To:
I had this problem as well.  As a workaround it helped to delete the old
index:

	rm  /usr/share/man/index_*

And build a new one:

	perlindex -verbose -index

Regards,
 Michael

#576810#51
Date:
2015-11-17 08:03:54 UTC
From:
To:
The index files are now in /var/cache/perlindex

I had similar trouble after installing libfile-slurper-perl.

But I could *not* reproduce the issue after the following steps:
- remove libfile-slurper-perl
- index
- install libfile-slurper-perl
- index

The second step complained about unvalid data fed to unpack during garbage
collection (line 170 of perlindex). Unfortunately, I've lost the actual error
message

I also noticed that the index was missing entries.

I ended cleaning up the index as suggested and rebuilt the index from scratch.

I think my index files somehow got corrupted. May be, purging the index
periodically should be suggested to user (or added in a cron job).

HTH