root filesystem: /dev/sda3 / btrfs rw,relatime,compress=zstd:1,ssd,
space_cache=v2,subvolid=256,subvol=/@rootfs 0 0
libc6: 2.36-9+deb12u1
kernel: 6.1.38-4
Kernel taint flags: TAINT_PROPRIETARY_MODULE, TAINT_OOT_MODULE,
TAINT_UNSIGNED_MODULE
I believe that fuser(1) should work on libraries, but on this system
it doesn't:
$ fuser /usr/lib/x86_64-linux-gnu/libc.so.6
$ stat -L /usr/lib/x86_64-linux-gnu/libc.so.6
File: /usr/lib/x86_64-linux-gnu/libc.so.6
Size: 1922136 Blocks: 3760 IO Block: 4096 regular file
Device: 0,27 Inode: 1834061 Links: 1
Access: (0755/-rwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2023-08-15 11:03:22.556128174 -0400
Modify: 2023-07-13 14:07:47.000000000 -0400
Change: 2023-07-24 23:46:40.235502863 -0400
Birth: 2023-07-24 23:46:39.943488643 -0400
$ grep -F libc.so /proc/$$/maps
7f6c81da2000-7f6c81dc8000 r--p 00000000 00:19 1834061 /usr/lib/x86_64-linux-gnu/libc.so.6
7f6c81dc8000-7f6c81f1d000 r-xp 00026000 00:19 1834061 /usr/lib/x86_64-linux-gnu/libc.so.6
7f6c81f1d000-7f6c81f70000 r--p 0017b000 00:19 1834061 /usr/lib/x86_64-linux-gnu/libc.so.6
7f6c81f70000-7f6c81f74000 r--p 001ce000 00:19 1834061 /usr/lib/x86_64-linux-gnu/libc.so.6
7f6c81f74000-7f6c81f76000 rw-p 001d2000 00:19 1834061 /usr/lib/x86_64-linux-gnu/libc.so.6
It looks like there is a disagreement about which device the file's on:
0,27 vs. 00:19 (i.e., 0,25).
If I change the selection criterion in fuser's check_map() to skip the
device check, then I get the sort of results I expect (although of
course it is wrong to do so).
--- src/fuser.c~ Tue Dec 13 00:22:55 2022
+++ src/fuser.c Tue Aug 15 18:47:33 2023
@@ -1760,8 +1760,12 @@ static void check_map(
if (dev_tmp->device == tmp_device)
add_matched_proc(dev_tmp->name, pid, uid, access);
for (ino_tmp = ino_head; ino_tmp != NULL; ino_tmp = ino_tmp->next)
- if (ino_tmp->device == tmp_device
+# if 0
+ if (ino_tmp->device == tmp_device
&& ino_tmp->inode == tmp_inode)
+# else
+ if (ino_tmp->inode == tmp_inode)
+# endif
add_matched_proc(ino_tmp->name, pid, uid, access);
}
}
Hi Paul, I don't think it is the libraries, the issue is the device IDs are different. [...] That is the issue. There's nothing fuser can really do about this. You asked it to find files on device 27, inode 1834061 and the program has mapped libc on device 25 inode 1834061. As far as fuser is concerned, this is like asking for a file on sda and trying to match it to something on sdb. It sure is! Again, that would match a file on sda to a file on sdb with the same inode. So, why is the device id different? It might be something odd with that filesystem, but it could be mount namespaces. Do you know if you have those? What does grep -e ' 0:2[57] ' /proc/self/mountinfo show? You could also try sudo nsenter -a -t <pid of process> fuser /usr/lib/x86_64-linux-gnu/libc.so.6 - Craig
/ is btrfs with just the single subvolume (to my knowledge), set up by the Debian installer. # btrfs subvolume list -a -t / ID gen top level path -- --- --------- ---- 256 42766 5 <FS_TREE>/@rootfs 29 1 0:25 /@rootfs / rw,relatime shared:1 - btrfs /dev/sda3 rw,compress=zstd:1,ssd,space_cache=v2,subvolid=256,subvol=/@rootfs It looks like interpreting the device number may be a problem with btrfs: https://lwn.net/Articles/866582/ "... the filesystem allocates a separate device number (the usual major/minor pair) for each subvolume; that number can be seen with a system call like stat(). [... Those] numbers do not show up in files like /proc/self/mountinfo, leading to inconsistent views of how the filesystem is put together." This doesn't seem to produce any output.
FWIW, fuser from psmisc-23.4 configured with "--enable-mountinfo-list" produces the kind of output I expect. (Without "--enable-mountinfo-list", it doesn't.)
I was looking at https://gitlab.com/psmisc/psmisc/-/issues/39 which is a similar, but different issue. Same physical device but due to mount namespaces different device ID. I'd be curious if lsof finds the access. Also the mountinfo-list finding it is curious, it matches devices a different way. - Craig
$ lsof /usr/lib/x86_64-linux-gnu/libc.so.6 $ lsof | grep -F /usr/lib/x86_64-linux-gnu/libc.so.6 | tail -n 1 lsof 56367 kimoto mem REG 0,25 1834061 /usr/lib/x86_64-linux-gnu/libc.so.6 (path dev=0,27) (Note the somewhat unusual "path dev=..." bit.) Maybe https://github.com/lsof-org/lsof/issues/152 is related.
I wrote the above, but it is just wrong. Sorry for the misinformation. -Paul