#783963 tracker-extract runs concurrently with updatedb.mlocate when opening Gnome session

Package:
tracker
Source:
tracker
Description:
metadata database, indexer and search tool
Submitter:
Ludovic Pouzenc
Date:
2015-05-30 17:30:03 UTC
Severity:
normal
#783963#5
Date:
2015-05-01 16:37:36 UTC
From:
To:
Dear Maintainer,

Right after booting and logging into a fresh default install of Debian 8
with a good quantity of files imported into ~/Documents/ my system came
very slow. This seems to happens roughly on each boot.

Opening a terminal takes 2 or 3 seconds. "top" says :

 PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
2156 lpouzenc  39  19 1723740  89952  32812 D  16,6  2,2   0:59.10 tracker-extract
2159 lpouzenc  20   0  479944  33744   8888 S  12,3  0,8   1:25.00 tracker-store
2233 lpouzenc  20   0  130956   6284   5348 S   1,3  0,2   0:07.84 gvfsd-metadata
2439 lpouzenc  20   0  421344  30744  22224 S   1,3  0,8   0:02.05 gnome-terminal-
2896 root      20   0    6904   2460   1856 D   1,3  0,1   0:04.77 updatedb.mlocat
 812 root      20   0  250200  45012  23656 S   0,7  1,1   0:05.02 Xorg

updatedb.mlocate eats IO, tracker-extract too, and when started
together everything goes slow, heads of my HDD go crazy.

Please try to see what you could do for preventing multiples file
crawler programs to be ran concurrently. If you cannot make simple
things, around that just defer a bit tracker update upon session
start. À là Windows (tm).

#783963#10
Date:
2015-05-12 20:50:39 UTC
From:
To:
Am 01.05.2015 um 18:37 schrieb Ludovic Pouzenc:

I don't see a good way to do this in a clean way. But I'm open to
suggestions. Maybe the

 If you cannot make simple

Doesn't help. updatedb.mlocate is triggered by a cron job (and we
already delay the start of the miner a bit after session start).

#783963#15
Date:
2015-05-27 15:18:19 UTC
From:
To:
]] Michael Biebl

This sentence seems cut off?

I'd be ok with having some way of signalling that updatedb is running so
other processes can quiesce themselves, if that sounds ok to you? (Also,
updatedb runs under ionice, not sure if tracker does the same thing?)

#783963#20
Date:
2015-05-30 16:45:50 UTC
From:
To:
Hi all and thanks for replies.

It seems that trackers already runs under ionice (at least on my machine)
.
 > 2146 lpouzenc  20   0  465412  15988   8168 S  96,6  0,4 0:05.16
tracker-store
 > 2132 lpouzenc  39  19 1104792  19016  11496 R  58,0  0,5 0:05.68
tracker-miner-f

 >lpouzenc@lud-GB1:~$ ionice -p 2146 2132
 > idle
 > idle

I was thinking to suggest using pid file to signal existance of other
"crawlers" instances. For /var/run, Debian FHS says <program-name.pid>,
so may use a generic folder like /var/run/crawlers/ or so. /var/lock is
more apropriate ? Already rwxrwxrwx although.

 >
https://www.debian.org/doc/packaging-manuals/fhs/fhs-2.3.html#VARLOCKLOCKFILES
 > Lock files for devices and other resources shared by multiple
applications, such as the serial device lock files that were originally
found in either /usr/spool/locks or /usr/spool/uucp, must now be stored
in /var/lock.

There is probably a problem about the ownership of the suggestion
subfolder (which package creates it, which user/group/rights on the folder).

strace suggest that mlocate protect itself from parallel exec by locking
his db file.
 > open("/var/lib/mlocate/mlocate.db", O_RDWR) = 4
 > read(4, "\0mlocate\0\0\0\370\0\1\0\0/\0prune_bind_mou"..., 8192) = 8192
 > fcntl(4, F_SETLK, {type=F_WRLCK, whence=SEEK_SET, start=0, len=0}) =
-1 EAGAIN (Resource temporarily unavailable)

tracker-control opens all /proc/*/cmdline to find his friends ? Sounds
weird.

 > lpouzenc@lud-GB1:~$ strace -ff -e lstat tracker-control 2>&1
 > Found 210 PIDs…
 > Process 3210 attached
 > [pid  3209] lstat("/proc/3209/cmdline", {st_mode=S_IFREG|0444,
st_size=0, ...}) = 0
 > [pid  3209] lstat("/proc/1/cmdline", {st_mode=S_IFREG|0444,
st_size=0, ...}) = 0
 > [pid  3209] lstat("/proc/2/cmdline", {st_mode=S_IFREG|0444,
st_size=0, ...}) = 0
 > [pid  3209] lstat("/proc/3/cmdline", {st_mode=S_IFREG|0444,
st_size=0, ...}) = 0
 > [pid  3209] lstat("/proc/5/cmdline", {st_mode=S_IFREG|0444,
st_size=0, ...}) = 0
 > [pid  3209] lstat("/proc/7/cmdline", {st_mode=S_IFREG|0444,
st_size=0, ...}) = 0

It seems that updatedb.mlocate and tracker does not use any pid file..
so I may go wrong :)

Why tracker don't use PID's ? It about the "random" user of the spawned
processes ?

Potential troll : Is systemd could solve that ? (I know that tracker is
started via xdg autostart and updatedb via cron)

Regards,
Ludovic