#728310 updatedb causes segfault mhddfs

Package:
mhddfs
Source:
mhddfs
Description:
file system for unifying several mount points into one
Submitter:
Victor Kraev
Date:
2014-08-27 04:51:10 UTC
Severity:
normal
#728310#5
Date:
2013-10-30 13:50:26 UTC
From:
To:
segfault happens when you run updatedb while rsync makes copy of files from
another server to mhddfs mount point and makes this last several hours.


root@ascella:~# grep mhddfs /var/log/messages
Oct 28 03:33:25 ascella kernel: [224706.970330] mhddfs[61772] general
protection ip:40443c sp:7f7b7d03b9d0 error:0 in mhddfs[400000+a000]
Oct 28 06:25:09 ascella kernel: [235006.128040] mhddfs[63139]: segfault at
1 ip 000000000040443c sp 00007f95a111b9d0 error 4 in mhddfs[400000+a000]
Oct 29 01:29:01 ascella kernel: [303609.942900] mhddfs[59593]: segfault at
fffffffffffffff9 ip 00007f4969aefa59 sp 00007f4968a709c0 error 5 in
libc-2.13.so[7f4969a74000+180000]
Oct 30 06:25:30 ascella kernel: [407754.912848] mhddfs[34600]: segfault at
fffffffffffffff9 ip 00007facc3f4ba59 sp 00007facc1c259c0 error 5 in
libc-2.13.so[7facc3ed0000+180000]
Oct 30 13:30:15 ascella kernel: [433229.084249] mhddfs[38870]: segfault at
0 ip 000000000040443c sp 00007fb7dd8cc9d0 error 4 in mhddfs[400000+a000]

last 20 lines of mhddfs.log before last crash:

grep -B20 RASH /var/log/mhddfs.log
mhddfs [2013-10-30 13:30:15] (info): [140427664205568] mhdd_write:
/torrents/Art - 2011/.Art - 2011-12-13 - 001 (x125) 2592x3888.zip.UOm2Ip,
handle = 140427579636560
mhddfs [2013-10-30 13:30:15] (debug): [140427664205568] < unlock flist
mhddfs [2013-10-30 13:30:15] (info): [140427681408768] mhdd_write:
/torrents/Art - 2011/.Art - 2011-12-13 - 001 (x125) 2592x3888.zip.UOm2Ip,
handle = 140427579636560
mhddfs [2013-10-30 13:30:15] (debug): [140427681408768] < unlock flist
mhddfs [2013-10-30 13:30:15] (info): [140427699128064] mhdd_write:
/torrents/Art - 2011/.Art - 2011-12-13 - 001 (x125) 2592x3888.zip.UOm2Ip,
handle = 140427579636560
mhddfs [2013-10-30 13:30:15] (debug): [140427699128064] < unlock flist
mhddfs [2013-10-30 13:30:15]: [140427681408768] mhdd_access:
/albums/main/210x260/6000 mode = 0001
mhddfs [2013-10-30 13:30:15] (info): [140427690735360] mhdd_write:
/torrents/Art - 2011/.Art - 2011-12-13 - 001 (x125) 2592x3888.zip.UOm2Ip,
handle = 140427579636560
mhddfs [2013-10-30 13:30:15]: [140427672737536] mhdd_stat:
/albums/main/210x260/6000/6380
mhddfs [2013-10-30 13:30:15] (debug): [140427690735360] < unlock flist
mhddfs [2013-10-30 13:30:15]: [140427664205568] mhdd_access:
/albums/main/210x260/6000/6380 mode = 0001
mhddfs [2013-10-30 13:30:15] (info): [140427699128064] mhdd_write:
/torrents/Art - 2011/.Art - 2011-12-13 - 001 (x125) 2592x3888.zip.UOm2Ip,
handle = 140427579636560
mhddfs [2013-10-30 13:30:15] (debug): [140427699128064] < unlock flist
mhddfs [2013-10-30 13:30:15]: [140427672737536] mhdd_readdir:
/albums/main/210x260/6000/6380
mhddfs [2013-10-30 13:30:15] (info): [140427690735360] mhdd_write:
/torrents/Art - 2011/.Art - 2011-12-13 - 001 (x125) 2592x3888.zip.UOm2Ip,
handle = 140427579636560
mhddfs [2013-10-30 13:30:15] (debug): [140427690735360] < unlock flist
mhddfs [2013-10-30 13:30:15] (info): [140427664205568] mhdd_write:
/torrents/Art - 2011/.Art - 2011-12-13 - 001 (x125) 2592x3888.zip.UOm2Ip,
handle = 140427579636560
mhddfs [2013-10-30 13:30:15] (debug): [140427664205568] < unlock flist
mhddfs [2013-10-30 13:30:15] (info): [140427681408768] mhdd_write:
/torrents/Art - 2011/.Art - 2011-12-13 - 001 (x125) 2592x3888.zip.UOm2Ip,
handle = 140427579636560
mhddfs [2013-10-30 13:30:15] (debug): [140427681408768] < unlock flist
CRASH

hdd and mount options configuration:

# df -h
Filesystem                                              Size  Used Avail
Use% Mounted on
rootfs                                                  105G  4.6G   95G
5% /
udev                                                     10M     0   10M
0% /dev
tmpfs                                                   2.4G  228K  2.4G
1% /run
/dev/disk/by-uuid/d87762c2-21ef-44ef-88c1-1485ff63cd19  105G  4.6G   95G
5% /
tmpfs                                                   5.0M     0  5.0M
0% /run/lock
tmpfs                                                   5.5G     0  5.5G
0% /run/shm
/dev/sda2                                                90M   17M   68M
 20% /boot
/dev/sda6                                               1.9G   35M  1.8G
2% /tmp
/dev/sdb1                                               2.7T  352G  2.4T
 13% /mnt/store1
/dev/sdc1                                               2.7T  263G  2.5T
 10% /mnt/store2
/dev/sdd1                                               2.7T  262G  2.5T
 10% /mnt/store3
/dev/sde1                                               2.7T  123G  2.5T
5% /mnt/store4
/mnt/store1;/mnt/store2;/mnt/store3;/mnt/store4          11T  999G  9.7T
 10% /data


# mount
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
udev on /dev type devtmpfs
(rw,relatime,size=10240k,nr_inodes=3086786,mode=755)
devpts on /dev/pts type devpts
(rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,noexec,relatime,size=2470632k,mode=755)
/dev/disk/by-uuid/d87762c2-21ef-44ef-88c1-1485ff63cd19 on / type ext4
(rw,relatime,errors=remount-ro,user_xattr,barrier=1,stripe=64,data=ordered)
tmpfs on /run/lock type tmpfs (rw,nosuid,nodev,noexec,relatime,size=5120k)
tmpfs on /run/shm type tmpfs (rw,nosuid,nodev,noexec,relatime,size=5741200k)
/dev/sda2 on /boot type ext2 (rw,relatime,errors=continue)
/dev/sda6 on /tmp type ext4
(rw,nosuid,nodev,noexec,relatime,user_xattr,barrier=1,stripe=64,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
/dev/sdb1 on /mnt/store1 type ext4
(rw,noatime,nodiratime,user_xattr,barrier=1,stripe=64,data=ordered)
/dev/sdc1 on /mnt/store2 type ext4
(rw,noatime,nodiratime,user_xattr,barrier=1,stripe=64,data=ordered)
/dev/sdd1 on /mnt/store3 type ext4
(rw,noatime,nodiratime,user_xattr,barrier=1,stripe=64,data=ordered)
fusectl on /sys/fs/fuse/connections type fusectl (rw,relatime)
nfsd on /proc/fs/nfsd type nfsd (rw,relatime)
/dev/sde1 on /mnt/store4 type ext4
(rw,noatime,nodiratime,user_xattr,barrier=1,stripe=64,data=ordered)
/mnt/store1;/mnt/store2;/mnt/store3;/mnt/store4 on /data type fuse.mhddfs
(rw,noatime,user_id=0,group_id=0,allow_other)

# cat /etc/stab

/dev/sdb1 /mnt/store1 ext4 defaults,noatime,nodiratime 0 0
/dev/sdc1 /mnt/store2 ext4 defaults,noatime,nodiratime 0 0
/dev/sdd1 /mnt/store3 ext4 defaults,noatime,nodiratime 0 0
/dev/sde1 /mnt/store4 ext4 defaults,noatime,nodiratime 0 0
mhddfs#/mnt/store1,/mnt/store2,/mnt/store3,/mnt/store4 /data fuse
defaults,allow_other,mlimit=3000G,loglevel=0,logfile=/var/log/mhddfs.log,noatime
0 0

Also /data uses by hfs kernel server and configured /etc/exports:

/data $PRIVATE_IP$(rw,async,fsid=0,no_subtree_check)

and /etc/fstab on client:

$PRIVATE_IP$:/data /mnt/storage1 nfs rw,async

if mlocate is installed then it does update of db daily at 6.25 by cron and
can cause segfault every day in stable debian. So option "fuse.mhddfs"
should be added to PRUNEFS in /etc/updatedb.conf for quick fix.

Let me know if you need more info.

#728310#10
Date:
2014-07-01 15:13:33 UTC
From:
To:
[1105212.935387] traps: mhddfs[27439] general protection ip:7f78a7dfcac9
sp:7f78a757e9c0 error:0 in libc-2.13.so[7f78a7d81000+182000]

This happens when rsync'ing to a mhddfs from a cifs share.

#728310#15
Date:
2014-08-27 04:41:21 UTC
From:
To:
I have been trying mhddfs on a samba file server the last several
months; at first it crashed daily, even with the change to update.conf
mentioned above, until I removed mlocate from the system. after that it
crashed weekly, until I changed the rsync schedule because it was
overlapping with workstations connecting.  after that it crashed every
10-14 days until I removed all packages related to nfs.  Since then it
was solid for nearly a month, but it crashed again today.

As best as I can tell nothing was happening except workstations
connecting.  The backup server was not rsyncing at the time, nothing
from cron runs at that time, there are no usb drives or any other
peripherals attached beyond keyboard and monitor, and I can find nothing
in the logs that appears to have happened within 5 minutes either side
of the event.

kernel: [1134938.097878] mhddfs[4406] general protection ip:7f583fccd824
sp:7f5825431a38 error:0 in libc-2.19.so[7f583fc51000+19f000]

If I can provide further info please say so...