#732072 openafs-modules-dkms: openafs loses connection, backtrace on restart

#732072#5
Date:
2013-12-13 15:31:39 UTC
From:
To:
I have a recent new install of Debian Wheezy on a 64bit AMD Opteron box and
have openafs-client installed.  Openafs does work and I can access the afs
directories.  After a certain amount of timem, the afs directories are not
accessible anymore even after a klog.  As root, I restart afs using /etc/init.d
/openafs-client restart.  I then see a backtrace for afs in syslog.  It appears
that openafs-client starts again because after a klog I can access the afs
directories again.

1. Is it normal for openafs-client to have to be restarted on a machine that is
not shut down very often?

2. That can't be good for it to backtrace in restart.

Here is the backtrace from the syslog:

Dec 13 10:13:18 opt16prof kernel: [63710.857297] afs: COLD shutting down of:
vcaches... CB... afs... BkG... CTrunc... AFSDB... RxEvent...
UnmaskRxkSignals... RxListener...  ALL allocated tables... done
Dec 13 10:13:18 opt16prof kernel: [63710.954593] enabling dynamically allocated
vcaches
Dec 13 10:13:18 opt16prof kernel: [63710.954598] Starting AFS cache
scan...found 21 non-empty cache files (1%).
Dec 13 10:13:18 opt16prof kernel: [63710.976641] ------------[ cut here
]------------
Dec 13 10:13:18 opt16prof kernel: [63710.976649] WARNING: at /build/linux-
rrsxby/linux-3.2.51/fs/sysfs/dir.c:481 sysfs_add_one+0x8d/0xa0()
Dec 13 10:13:18 opt16prof kernel: [63710.976653] Hardware name: -[6217Z03]-
Dec 13 10:13:18 opt16prof kernel: [63710.976654] sysfs: cannot create duplicate
filename '/devices/virtual/bdi/afs'
Dec 13 10:13:18 opt16prof kernel: [63710.976657] Modules linked in: openafs(P)
tcp_diag inet_diag ppdev lp bnep ipt_LOG ipt_REJECT iptable_nat nf_nat
nf_conntrack_ipv4 nf_defrag_ipv4 rfcomm bluetooth rfkill iptable_filter
ip_tables ip6t_LOG xt_limit ip6t_REJECT xt_tcpudp ip6t_ah nf_conntrack_ipv6
nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables x_tables
binfmt_misc nfsd nfs nfs_acl auth_rpcgss fscache lockd sunrpc ext2 loop
nvidia(P) parport_pc parport shpchp psmouse mperf i2c_amd756 amd_rng
amd64_edac_mod container rng_core edac_mce_amd snd_intel8x0 snd_ac97_codec
evdev i2c_amd8111 i2c_core pcspkr serio_raw k8temp edac_core snd_pcm
snd_page_alloc snd_seq snd_seq_device snd_timer snd processor button soundcore
ac97_bus thermal_sys ext4 crc16 jbd2 mbcache dm_mod microcode sg sd_mod
crc_t10dif sr_mod cdrom ata_generic ohci_hcd ehci_hcd floppy sata_sil pata_amd
aic79xx scsi_transport_spi usbcore firewire_ohci tg3 libphy usb_common libata
firewire_core crc_itu_t scsi_mod [last unloaded: openafs]
Dec 13 10:13:18 opt16prof kernel: [63710.976735] Pid: 14955, comm: afsd
Tainted: P           O 3.2.0-4-amd64 #1 Debian 3.2.51-1
Dec 13 10:13:18 opt16prof kernel: [63710.976737] Call Trace:
Dec 13 10:13:18 opt16prof kernel: [63710.976744]  [<ffffffff81046cbd>] ?
warn_slowpath_common+0x78/0x8c
Dec 13 10:13:18 opt16prof kernel: [63710.976748]  [<ffffffff81046d69>] ?
warn_slowpath_fmt+0x45/0x4a
Dec 13 10:13:18 opt16prof kernel: [63710.976751]  [<ffffffff81150575>] ?
sysfs_add_one+0x8d/0xa0
Dec 13 10:13:18 opt16prof kernel: [63710.976755]  [<ffffffff811505ee>] ?
create_dir+0x66/0xa0
Dec 13 10:13:18 opt16prof kernel: [63710.976758]  [<ffffffff811506c3>] ?
sysfs_create_dir+0x85/0x9b
Dec 13 10:13:18 opt16prof kernel: [63710.976763]  [<ffffffff810bb3cf>] ?
__alloc_pages_nodemask+0x186/0x7aa
Dec 13 10:13:18 opt16prof kernel: [63710.976768]  [<ffffffff811abe22>] ?
kobject_add_internal+0xc8/0x181
Dec 13 10:13:18 opt16prof kernel: [63710.976771]  [<ffffffff811ac1f7>] ?
kobject_add+0x95/0xa4
Dec 13 10:13:18 opt16prof kernel: [63710.976775]  [<ffffffff8134e6cd>] ?
mutex_lock+0xd/0x2d
Dec 13 10:13:18 opt16prof kernel: [63710.976779]  [<ffffffff8124f7ef>] ?
device_add+0x109/0x590
Dec 13 10:13:18 opt16prof kernel: [63710.976783]  [<ffffffff8125976f>] ?
pm_runtime_init+0xb5/0xc9
Dec 13 10:13:18 opt16prof kernel: [63710.976786]  [<ffffffff8124fd0e>] ?
device_create_vargs+0x86/0xbe
Dec 13 10:13:18 opt16prof kernel: [63710.976790]  [<ffffffff810cb7c8>] ?
bdi_register+0x5f/0x18b
Dec 13 10:13:18 opt16prof kernel: [63710.976811]  [<ffffffffa0d4a52d>] ?
afs_fill_super+0x17d/0x20d [openafs]
Dec 13 10:13:18 opt16prof kernel: [63710.976820]  [<ffffffffa0d4a3b0>] ?
afs_root.isra.1+0x131/0x131 [openafs]
Dec 13 10:13:18 opt16prof kernel: [63710.976825]  [<ffffffff810fc731>] ?
mount_nodev+0x42/0x78
Dec 13 10:13:18 opt16prof kernel: [63710.976828]  [<ffffffff810fd117>] ?
mount_fs+0x61/0x146
Dec 13 10:13:18 opt16prof kernel: [63710.976832]  [<ffffffff811101a6>] ?
vfs_kern_mount+0x5f/0x99
Dec 13 10:13:18 opt16prof kernel: [63710.976835]  [<ffffffff81110590>] ?
do_kern_mount+0x49/0xd8
Dec 13 10:13:18 opt16prof kernel: [63710.976838]  [<ffffffff81111c0b>] ?
do_mount+0x660/0x6c6
Dec 13 10:13:18 opt16prof kernel: [63710.976842]  [<ffffffff810c9858>] ?
memdup_user+0x36/0x5b
Dec 13 10:13:18 opt16prof kernel: [63710.976845]  [<ffffffff81111f19>] ?
sys_mount+0x88/0xc3
Dec 13 10:13:18 opt16prof kernel: [63710.976849]  [<ffffffff81354212>] ?
system_call_fastpath+0x16/0x1b
Dec 13 10:13:18 opt16prof kernel: [63710.976852] ---[ end trace
69aa9c0cb4b3d069 ]---
Dec 13 10:13:18 opt16prof kernel: [63710.976856] kobject_add_internal failed
for afs with -EEXIST, don't try to register things with the same name in the
same directory.
Dec 13 10:13:18 opt16prof kernel: [63710.976861] Pid: 14955, comm: afsd
Tainted: P        W  O 3.2.0-4-amd64 #1 Debian 3.2.51-1
Dec 13 10:13:18 opt16prof kernel: [63710.976864] Call Trace:
Dec 13 10:13:18 opt16prof kernel: [63710.976867]  [<ffffffff811abeb1>] ?
kobject_add_internal+0x157/0x181
Dec 13 10:13:18 opt16prof kernel: [63710.976870]  [<ffffffff811ac1f7>] ?
kobject_add+0x95/0xa4
Dec 13 10:13:18 opt16prof kernel: [63710.976873]  [<ffffffff8134e6cd>] ?
mutex_lock+0xd/0x2d
Dec 13 10:13:18 opt16prof kernel: [63710.976877]  [<ffffffff8124f7ef>] ?
device_add+0x109/0x590
Dec 13 10:13:18 opt16prof kernel: [63710.976880]  [<ffffffff8125976f>] ?
pm_runtime_init+0xb5/0xc9
Dec 13 10:13:18 opt16prof kernel: [63710.976883]  [<ffffffff8124fd0e>] ?
device_create_vargs+0x86/0xbe
Dec 13 10:13:18 opt16prof kernel: [63710.976886]  [<ffffffff810cb7c8>] ?
bdi_register+0x5f/0x18b
Dec 13 10:13:18 opt16prof kernel: [63710.976896]  [<ffffffffa0d4a52d>] ?
afs_fill_super+0x17d/0x20d [openafs]
Dec 13 10:13:18 opt16prof kernel: [63710.976905]  [<ffffffffa0d4a3b0>] ?
afs_root.isra.1+0x131/0x131 [openafs]
Dec 13 10:13:18 opt16prof kernel: [63710.976909]  [<ffffffff810fc731>] ?
mount_nodev+0x42/0x78
Dec 13 10:13:18 opt16prof kernel: [63710.976912]  [<ffffffff810fd117>] ?
mount_fs+0x61/0x146
Dec 13 10:13:18 opt16prof kernel: [63710.976915]  [<ffffffff811101a6>] ?
vfs_kern_mount+0x5f/0x99
Dec 13 10:13:18 opt16prof kernel: [63710.976918]  [<ffffffff81110590>] ?
do_kern_mount+0x49/0xd8
Dec 13 10:13:18 opt16prof kernel: [63710.976921]  [<ffffffff81111c0b>] ?
do_mount+0x660/0x6c6
Dec 13 10:13:18 opt16prof kernel: [63710.976925]  [<ffffffff810c9858>] ?
memdup_user+0x36/0x5b
Dec 13 10:13:18 opt16prof kernel: [63710.976928]  [<ffffffff81111f19>] ?
sys_mount+0x88/0xc3
Dec 13 10:13:18 opt16prof kernel: [63710.976931]  [<ffffffff81354212>] ?
system_call_fastpath+0x16/0x1b

#732072#10
Date:
2013-12-13 18:55:27 UTC
From:
To:
"Brent S. Elmer Ph.D." <webe3vt@aim.com> writes:

No.

The backtrace is because AFS was never shut down properly and is trying to
re-register things that weren't cleaned up.  That's just another
consequence of the fact that AFS was in some sort of inconsistent state.

Unfortunately, there's no data here to know why AFS got into an
inconsistent state to start with.  Are there any symptoms at all beyond
AFS directories not being accessible?  Is this consistently reproducible
on this system?

Does using the newer version of openafs-client and openafs-modules-dkms
from backports.debian.org help?

#732072#15
Date:
2013-12-13 18:55:27 UTC
From:
To:
"Brent S. Elmer Ph.D." <webe3vt@aim.com> writes:

No.

The backtrace is because AFS was never shut down properly and is trying to
re-register things that weren't cleaned up.  That's just another
consequence of the fact that AFS was in some sort of inconsistent state.

Unfortunately, there's no data here to know why AFS got into an
inconsistent state to start with.  Are there any symptoms at all beyond
AFS directories not being accessible?  Is this consistently reproducible
on this system?

Does using the newer version of openafs-client and openafs-modules-dkms
from backports.debian.org help?

#732072#20
Date:
2013-12-13 20:01:13 UTC
From:
To:
No, I don't notice any other symptoms.
yes, it seems to happen every day.

I have installed the newer version from backports.  I'll watch it and
see if that helps.  By the way, would installing openafs from testing on
a wheezy system be much different than installing from backports?

Brent

#732072#25
Date:
2013-12-13 20:01:13 UTC
From:
To:
No, I don't notice any other symptoms.
yes, it seems to happen every day.

I have installed the newer version from backports.  I'll watch it and
see if that helps.  By the way, would installing openafs from testing on
a wheezy system be much different than installing from backports?

Brent

#732072#30
Date:
2013-12-13 20:17:54 UTC
From:
To:
"Brent S. Elmer Ph.D." <webe3vt@aim.com> writes:

When AFS stops working, do attempts to access AFS hang, or do you get
permission denied errors, or something else?

Is there anything in the kernel log (dmesg)?

No, they should be equivalent.  The only difference in wheezy is support
for newer kernels.

#732072#35
Date:
2013-12-13 20:17:54 UTC
From:
To:
"Brent S. Elmer Ph.D." <webe3vt@aim.com> writes:

When AFS stops working, do attempts to access AFS hang, or do you get
permission denied errors, or something else?

Is there anything in the kernel log (dmesg)?

No, they should be equivalent.  The only difference in wheezy is support
for newer kernels.

#732072#40
Date:
2013-12-16 15:07:46 UTC
From:
To:
It doesn't hang.  I think it was permission denied errors.

No, other than the backtrace I already showed you when I did a restart.

After installing the later version from backports on Friday, AFS is
still up and working today.  There wasn't much activity on the machine
since then either.  I will continue to watch it and see if it stays up.

Thanks,

Brent

#732072#45
Date:
2013-12-16 15:07:46 UTC
From:
To:
It doesn't hang.  I think it was permission denied errors.

No, other than the backtrace I already showed you when I did a restart.

After installing the later version from backports on Friday, AFS is
still up and working today.  There wasn't much activity on the machine
since then either.  I will continue to watch it and see if it stays up.

Thanks,

Brent