Hi. Since 0.7.45 I see a problem, that many deamons fail to start during boot, since some of the configured addresses are not yet ready, See the following bootlogd snipped, where apache, and bind fail: (Nothing has been logged yet.)$ Tue Oct 22 04:16:54 2013: [....] Setting parameters of disc: (none)^[[?25l^[[?1c^[7^[[1G[^[[32m ok ^[[39;49m^[8^[[?25h^[[?0c.$ Tue Oct 22 04:16:54 2013: [....] Setting preliminary keymap...^[[?25l^[[?1c^[7^[[1G[^[[32m ok ^[[39;49m^[8^[[?25h^[[?0cdone.$ Tue Oct 22 04:16:54 2013: [....] Activating swap...^[[?25l^[[?1c^[7^[[1G[^[[32m ok ^[[39;49m^[8^[[?25h^[[?0cdone.$ Tue Oct 22 04:16:54 2013: [....] Checking root file system...fsck from util-linux 2.20.1$ Tue Oct 22 04:16:54 2013: root: clean, 168884/30195712 files, 5681928/120753152 blocks$ Tue Oct 22 04:16:55 2013: ^[[?25l^[[?1c^[7^[[1G[^[[32m ok ^[[39;49m^[8^[[?25h^[[?0cdone.$ Tue Oct 22 04:16:55 2013: [....] Cleaning up temporary files... /tmp^[[?25l^[[?1c^[7^[[1G[^[[32m ok ^[[39;49m^[8^[[?25h^[[?0c.$ Tue Oct 22 04:16:55 2013: [^[[36minfo^[[39;49m] Loading kernel module fuse.$ Tue Oct 22 04:16:55 2013: [^[[36minfo^[[39;49m] Loading kernel module w83627ehf.$ Tue Oct 22 04:16:55 2013: [....] Generating udev events for MD arrays...^[[?25l^[[?1c^[7^[[1G[^[[32m ok ^[[39;49m^[8^[[?25h^[[?0cdone.$ Tue Oct 22 04:16:55 2013: [....] Starting early crypto disks...^[[?25l^[[?1c^[7^[[1G[^[[32m ok ^[[39;49m^[8^[[?25h^[[?0cdone.$ Tue Oct 22 04:16:55 2013: [....] Setting up LVM Volume Groups...^[[?25l^[[?1c^[7^[[1G[^[[32m ok ^[[39;49m^[8^[[?25h^[[?0cdone.$ Tue Oct 22 04:16:56 2013: [....] Starting remaining crypto disks...^[[?25l^[[?1c^[7^[[1G[^[[32m ok ^[[39;49m^[8^[[?25h^[[?0cdone.$ Tue Oct 22 04:16:56 2013: [....] Activating lvm and md swap...^[[?25l^[[?1c^[7^[[1G[^[[32m ok ^[[39;49m^[8^[[?25h^[[?0cdone.$ Tue Oct 22 04:16:56 2013: [....] Checking file systems...fsck from util-linux 2.20.1$ Tue Oct 22 04:16:56 2013: ^[[?25l^[[?1c^[7^[[1G[^[[32m ok ^[[39;49m^[8^[[?25h^[[?0cdone.$ Tue Oct 22 04:16:56 2013: [....] Mounting local filesystems...^[[?25l^[[?1c^[7^[[1G[^[[32m ok ^[[39;49m^[8^[[?25h^[[?0cdone.$ Tue Oct 22 04:16:56 2013: [....] Activating swapfile swap...^[[?25l^[[?1c^[7^[[1G[^[[32m ok ^[[39;49m^[8^[[?25h^[[?0cdone.$ Tue Oct 22 04:16:56 2013: [....] Cleaning up temporary files...^[[?25l^[[?1c^[7^[[1G[^[[32m ok ^[[39;49m^[8^[[?25h^[[?0c.$ Tue Oct 22 04:16:56 2013: [....] Setting kernel variables ...^[[?25l^[[?1c^[7^[[1G[^[[32m ok ^[[39;49m^[8^[[?25h^[[?0cdone.$ Tue Oct 22 04:16:58 2013: [....] Stopping authentication failure monitor: fail2ban^[[?25l^[[?1c^[7^[[1G[^[[32m ok ^[[39;49m^[8^[[?25h^[[?0c.$ Tue Oct 22 04:16:58 2013: [....] Loading iptables rules... IPv4... IPv6...^[[?25l^[[?1c^[7^[[1G[^[[32m ok ^[[39;49m^[8^[[?25h^[[?0cdone.$ Tue Oct 22 04:16:58 2013: [....] Setting up resolvconf...^[[?25l^[[?1c^[7^[[1G[^[[32m ok ^[[39;49m^[8^[[?25h^[[?0cdone.$ Tue Oct 22 04:16:58 2013: [....] Configuring network interfaces...ifup: interface eth0 already configured$ Tue Oct 22 04:16:59 2013: ^[[?25l^[[?1c^[7^[[1G[^[[32m ok ^[[39;49m^[8^[[?25h^[[?0cdone.$ Tue Oct 22 04:16:59 2013: [....] Starting rpcbind daemon...^[[?25l^[[?1c^[7^[[1G[^[[32m ok ^[[39;49m^[8^[[?25h^[[?0c.$ Tue Oct 22 04:16:59 2013: [....] Starting NFS common utilities:^[[?25l^[[?1c^[7^[[1G[^[[32m ok ^[[39;49m^[8^[[?25h^[[?0c.$ Tue Oct 22 04:16:59 2013: [....] Cleaning up temporary files...^[[?25l^[[?1c^[7^[[1G[^[[32m ok ^[[39;49m^[8^[[?25h^[[?0c.$ Tue Oct 22 04:16:59 2013: [^[[36minfo^[[39;49m] Setting console screen modes.$ Tue Oct 22 04:16:59 2013: ^[[9;30]^[[14;30][....] Setting up console font and keymap...^[[?25l^[[?1c^[7^[[1G[^[[32m ok ^[[39;49m^[8^[[?25h^[[?0cdone.$ Tue Oct 22 04:16:59 2013: [....] Setting up X socket directories... /tmp/.X11-unix /tmp/.ICE-unix^[[?25l^[[?1c^[7^[[1G[^[[32m ok ^[[39;49m^[8^[[?25h^[[?0c.$ Tue Oct 22 04:16:59 2013: Loading the saved-state of the serial devices... $ Tue Oct 22 04:16:59 2013: ... handled by kernel$ Tue Oct 22 04:16:59 2013: [....] Setting sensors limits^[[?25l^[[?1c^[7^[[1G[^[[32m ok ^[[39;49m^[8^[[?25h^[[?0c.$ Tue Oct 22 04:17:00 2013: [^[[36minfo^[[39;49m] zfs-fuse is disabled by /etc/default/zfs-fuse..$ Tue Oct 22 04:17:00 2013: [....] Loading IPsec SA/SP database: $ Tue Oct 22 04:17:00 2013: - /etc/ipsec-tools.conf^[[?25l^[[?1c^[7^[[1G[^[[32m ok ^[[39;49m^[8^[[?25h^[[?0c.$ Tue Oct 22 04:17:01 2013: INIT: Entering runlevel: 2$ Tue Oct 22 04:17:01 2013: [^[[36minfo^[[39;49m] Using makefile-style concurrent boot in runlevel 2.$ Tue Oct 22 04:17:01 2013: [....] Starting NFS common utilities:^[[?25l^[[?1c^[7^[[1G[^[[32m ok ^[[39;49m^[8^[[?25h^[[?0c.$ Tue Oct 22 04:17:01 2013: Not starting udftools packet writing: No devices listed in /etc/default/udftools$ Tue Oct 22 04:17:01 2013: [....] Not enabling Memory Error Detection and Correction since EDAC_DRIVER is not set:^[[?25l^[[?1c^[7^[[1G[^[[32m ok ^[[39;49m^[8^[[?25h^[[?0c.$ Tue Oct 22 04:17:01 2013: [....] Loading DIMM labels for Memory Error Detection and Correction: edac^[[?25l^[[?1c^[7^[[1G[^[[32m ok ^[[39;49m^[8^[[?25h^[[?0c.$ Tue Oct 22 04:17:01 2013: [....] Enabling additional executable binary formats: binfmt-support^[[?25l^[[?1c^[7^[[1G[^[[32m ok ^[[39;49m^[8^[[?25h^[[?0c.$ Tue Oct 22 04:17:01 2013: [....] Starting enhanced syslogd: rsyslogd^[[?25l^[[?1c^[7^[[1G[^[[32m ok ^[[39;49m^[8^[[?25h^[[?0c.$ Tue Oct 22 04:17:01 2013: [....] Starting ACPI services...RTNETLINK1 answers: No such file or directory$ Tue Oct 22 04:17:01 2013: acpid: error talking to the kernel via netlink$ Tue Oct 22 04:17:01 2013: ^[[?25l^[[?1c^[7^[[1G[^[[32m ok ^[[39;49m^[8^[[?25h^[[?0c.$ Tue Oct 22 04:17:01 2013: [....] Starting anac(h)ronistic cron: anacron^[[?25l^[[?1c^[7^[[1G[^[[32m ok ^[[39;49m^[8^[[?25h^[[?0c.$ Tue Oct 22 04:17:02 2013: [....] Starting deferred execution scheduler: atd^[[?25l^[[?1c^[7^[[1G[^[[32m ok ^[[39;49m^[8^[[?25h^[[?0c.$ Tue Oct 22 04:17:02 2013: [....] Starting acpi_fakekey daemon...^[[?25l^[[?1c^[7^[[1G[^[[32m ok ^[[39;49m^[8^[[?25h^[[?0cdone.$ Tue Oct 22 04:17:02 2013: [....] Enabling power management...^[[?25l^[[?1c^[7^[[1G[^[[32m ok ^[[39;49m^[8^[[?25h^[[?0cdone.$ Tue Oct 22 04:17:02 2013: [....] Starting system message bus: dbus^[[?25l^[[?1c^[7^[[1G[^[[32m ok ^[[39;49m^[8^[[?25h^[[?0c.$ Tue Oct 22 04:17:02 2013: [....] Starting domain name service...: bind9^[[?25l^[[?1c^[7^[[1G[^[[31mFAIL^[[39;49m^[8^[[?25h^[[?0c ^[[31mfailed!^[[39;49m$ Tue Oct 22 04:17:02 2013: [....] Starting authentication failure monitor: fail2ban^[[?25l^[[?1c^[7^[[1G[^[[32m ok ^[[39;49m^[8^[[?25h^[[?0c.$ Tue Oct 22 04:17:03 2013: [....] Starting web server: apache2(99)Cannot assign requested address: make_sock: could not bind to address [2a01:4f8:a0:4024::c:0]:80$ Tue Oct 22 04:17:04 2013: no listening sockets available, shutting down$ Tue Oct 22 04:17:04 2013: Unable to open logs$ Tue Oct 22 04:17:04 2013: Action 'start' failed.$ Tue Oct 22 04:17:04 2013: The Apache error log may have more information.$ Tue Oct 22 04:17:04 2013: ^[[?25l^[[?1c^[7^[[1G[^[[31mFAIL^[[39;49m^[8^[[?25h^[[?0c ^[[31mfailed!^[[39;49m$ Tue Oct 22 04:17:04 2013: [....] Starting MD monitoring service: mdadm --monitor^[[?25l^[[?1c^[7^[[1G[^[[32m ok ^[[39;49m^[8^[[?25h^[[?0c.$ Tue Oct 22 04:17:04 2013: Starting Machine Check Exceptions decoder: mcelog.$ Tue Oct 22 04:17:04 2013: [....] Starting NTP server: ntpd^[[?25l^[[?1c^[7^[[1G[^[[32m ok ^[[39;49m^[8^[[?25h^[[?0c.$ Tue Oct 22 04:17:04 2013: [....] Starting periodic command scheduler: cron^[[?25l^[[?1c^[7^[[1G[^[[32m ok ^[[39;49m^[8^[[?25h^[[?0c.$ Tue Oct 22 04:17:04 2013: [....] Loading cpufreq kernel modules...^[[?25l^[[?1c^[7^[[1G[^[[31mFAIL^[[39;49m^[8^[[?25h^[[?0c^[[31mfailed.^[[39;49m$ Tue Oct 22 04:17:04 2013: [....] Starting SMP IRQ Balancer: irqbalance^[[?25l^[[?1c^[7^[[1G[^[[32m ok ^[[39;49m^[8^[[?25h^[[?0c.$ Tue Oct 22 04:17:04 2013: [....] Starting disk temperature monitoring daemon: hddtemp: /dev/sda /dev/sdb^[[?25l^[[?1c^[7^[[1G[^[[32m ok ^[[39;49m^[8^[[?25h^[[?0c.$ Tue Oct 22 04:17:04 2013: [....] Starting Postfix Mail Transport Agent: postfix[....] CPUFreq Utilities: Setting ondemand CPUFreq governor...disabled, governor not available...^[[?25l^[[?1c^[7^[[1G[^[[32m ok ^[[39;49m^[8^[[?25h^[[?0cdone.$ Tue Oct 22 04:17:05 2013: Starting sks daemons: sksdb.. sksrecon.. done.$ Tue Oct 22 04:17:05 2013: ^[[?25l^[[?1c^[7^[[1G[^[[32m ok ^[[39;49m^[8^[[?25h^[[?0c.$ Tue Oct 22 04:17:05 2013: [....] Starting OpenBSD Secure Shell server: sshd^[[?25l^[[?1c^[7^[[1G[^[[32m ok ^[[39;49m^[8^[[?25h^[[?0c.$ Tue Oct 22 04:17:05 2013: [....] Starting sensor daemon: sensord^[[?25l^[[?1c^[7^[[1G[^[[32m ok ^[[39;49m^[8^[[?25h^[[?0c.$ Tue Oct 22 04:17:06 2013: [....] Setting sysfs variables...^[[?25l^[[?1c^[7^[[1G[^[[32m ok ^[[39;49m^[8^[[?25h^[[?0c.$ Going back to 0.7.44 fixes the problem. Cheers, Chris.
Hello, The major difference between 0.7.44 and 0.7.45 is that 0.7.45 waits for DAD to complete for 'inet6 static' interfaces. That shouldn't prevent the address from being assigned, however. Could you please get some more information from the system?
Well couldn't that be connected? Since my it seem to be exactly the inet6 addrs that are not _yet_ configured. (They are though configured, once I log in.) Sure... but nothing it special,.. what exactly would you like to see? Cheers, Chris
Hello, It's definitely connected, I can't understand how, however. When waiting for DAD, it says ‘Waiting for DAD...’, which I don't see in the logs you gave me. Also, DAD is usually quick, and anyway apache and bind shouldn't start until interfaces finished configuring, as interface configuration is blocking... unless it's initiated by udev. Do you have your eth0 as allow-hotplug? Try changing it to auto and see what happens. Also, please note the new dad-attempts parameter. If you set it to 0, it behaves as in 0.7.44.
Both fixes the problem ...either using allow-auto respectively auto or setting dad-attempts 0 for each interface, while continuing to use allow-hotplug Cheers, Chris.
--- Please enter the report below this line. ---
Updated now to 0.7.46-1. Anything new. Bug is a month old.
Related to #723184 posted around same time?
Debian Release: jessie/sid
500 wheezy linux.dropbox.com
500 unstable ftp.us.debian.org
500 unstable debian.qelectrotech.org
500 testing ftp.us.debian.org
500 precise ppa.launchpad.net
1 experimental ftp.us.debian.org
--- Package information. ---
Depends (Version) | Installed
================================-+-=================
iproute2 | 3.11.0-1
libc6 (>= 2.7) |
sysv-rc (>= 2.88dsf-24) |
OR file-rc (>= 0.8.16) |
lsb-base (>= 4.1+Debian3) |
initscripts (>= 2.88dsf-25) |
Recommends (Version) | Installed
==============================-+-===========
isc-dhcp-client | 4.2.4-7
OR dhcp-client |
Suggests (Version) | Installed
========================-+-===========
ppp | 2.4.5+git20130610-1
rdnssd |
net-tools | 1.60-25
Hello, Well, it's not strictly a bug... It's rather incorrect usage. If you use static IPv6 address assignment, you must wait for DAD to complete before using the address. If you use allow-hotplug, it brings the interface up asynchronously, and apache and other services start before DAD has completed. So basically there are two ways of solving the issue: * Use auto, not allow-hotplug, and leave DAD check enabled. * Disable DAD check; it's not guaranteed then IPv6 address will be ready when services start. No, unrelated. In that bug report, there's probably some kernel issue; the original bug posted still hasn't provided any details.
So a changes notification with instructions may be in order. Issue might not even apply to my setup, but if I do encounter it, where do I change the options?
Hello, The problem here is that I'm not sure what to do with this bug. I could patch udev's script which implements allow-hotplug so it would ignore inet6 static interfaces before ‘networking’ init script has run, but this won't solve the issue completely... If a service could depend on a interface being up, not on ‘networking’, that would make it much easier to do something with this. More thinking required.
Hey. Anything new here? First of all, I'm pretty unhappy with hiding away that bug behind severity "normal"? I mean that issue means, that with the default way of IPv6 used to be set up (i.e. eth0 being allow-hotplug) many services fail to work. So the proper severity by definition is critical as I've set it in the beginning. The only reason for not re-setting it is that I'm tired of these severity-wars. Second,... all this used to work until version 0.7.45, so there is a clear thing which has changed (0.7.45 waits for DAD to complete for 'inet6 static' interfaces) and it should be possible to decide what to do (or revert that change). Now you we've seen that it works with allow-auto, but not with allow-hotplug. To be honest, I've never fully understood what hotplug really does here, and I couldn't find any place where it's documented (at least not in interfaces(5) - probably something that should be done). If that would be documented and if that says "don't use hotplug with static addresses" than one could easily say "okay people have to change their setups". But OTOH it sounds kinda strange that allow-* would have an effect on whether dhcp or static. So what exactly is the difference that it can work with auto but not hotplug, when having DAD enabled in both cases? Is this just a timing issue? Or why does IPv6 not work with -hotplug, while it does with -auto? Would event-driven init-systems (i.e. systemd) help us here? Anyway, if no solution is found, then either this should be properly documented (with the ideal solution what people should do disable-DAD vs. -auto) or ifupdown should implicitly either disable DAD (probably a bad idea?) or implicitly convert the allow-hotplug to allow-auto. Cheers, Chris.
It is definitely severity normal. I have already explained how it works. If you want synchronous operation, don't use allow-hotplug. By its definition, it means asynchronous, which means your servers can't rely on those interfaces.
Well you have explained that here, but this is nothing that a admin necessarily reads before upgrading... he would only read it once his stuff already starts to fail. So as along as the default is still to have allow-hotplug on eth interfaces (which is still the case, AFAIK) and as long as there is no NEWS.Debian entry or other measure to warn people that update their ifupdown, that it will likely fail with IPv6 and their current setup - the issue still fully exists and is therefore still as severe (namely critical) as it was on the first day. Justification: it breaks unrelated software Okay but as said before, I don't quite understand what hotplug means, respectively what async/sync means in this case? Does it mean that ifupdown brings the interface up once the link comes up, even if this is at a later point? But, still,... what's the appropriate solution? Shall we change the default from allow-hotplug to allow-auto? Shall we try to find a solution with systemd? Cheers, Chris.
Hello, No, no and no. The default is allow-hotplug and auto, which means no DAD waiting. If you chose to change the setting to static, it's your job to ensure it actually works. Allow-hotplug means ‘I don't know and don't want to know and care when this interface comes up, and if it comes up at all, I have nothing that depends on it during the boot-up sequence, maybe I don't have it at all, but when it appears, I want it to be brought up’. If you want your interface to be brought during boot no matter what, don't use allow-hotplug, use auto.
Hey. There's also that issue:#768514 It's smells after one of these three bugs here (#766943, #727073 and #766291), but OTOH, the 30s sleep ugly workaround is still in place right now, and all other daemons can bind to their addresses,... bind9 however can not and fails at start. Cheers, Chris.
fixed 766291 0.7.50 fixed 727073 0.7.50 stop Hi Michael. Yes it's "fixed/workarounded". The same also applies to #727073, which his fixed by 0.7.50 as well. I'm not sure whether we should close this bug now, since the udevadm settle is only intended to be a temporary workaround, right? So maybe it would be good to keep those bugs open and just marked fixed in versions xxx (which I did above), so that once we properly get rid of the udevadm settle, all these issues can be checked back again. Cheers, Chris.