Hi Christoph,

clevis-initramfs emits the following warning at boot time for me (and
then seems to cease to work, which might be a followup error or a
separate problem):

  clevis: Warning: multiple network interfaces available but no ip= parameter provided.

But the host only has one NIC, both, from hardware inspection as well
as from the software side:

  $ ifconfig -a | egrep -v '^ |^$'
  eno1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
  lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536

I also see "eno1" in other boot messages, so it's probably not still
named "eth0" at that time. I also see that it got an IPv4 address via
DHCP on the third try. (Can provide a photograph of the bootup
messages if wanted.)

I saw this on Debian 11 Bullseye host, but also looked at the code from
18-1 as well. I've copied the clevis_all_netbootable_devices function
from both versions (16-2 and 18-1) into a shell script and ran it on the
booted host (prefixed the "echo" parameter with the package version as
well):

# ./clevis_all_netbootable_devices.sh
18-1: eno1
16-2: eno1
#

So I can't really understand why this warning is emitted.

Axel Beckert wrote...
(...)
hasn't changed, there's reason to assume the issue exists, also
upstream, and therefore I'd like to fix it.

However, looking at the code I fail to understand what's happening. Can
you help out a little (assuming the issue is still around).

This is a simple setup, I assume? So, pin is just one tang server?

The only explanation I can think of is the DEVICE variable is already
set by some means. So, does inserting "local DEVICE" in the first line
of clevis_all_netbootable_devices change something? Likewise some debug
prints around that.

Are some nonstandard initramfs scripts in the game that export DEVICE?

Does the DCHP discovery take an unsual amount of time, say: More than
five seconds?

Regards,

    Christoph

Hi Christoph,

TL;DR: It's never DNS! — It was DNS.

Christoph Biedl wrote:
[…]

Haven't tried for a while but since I haven't changed much of the
setup…

Please tell me in case I should rather use 18-2 from Testing instead
of 16-2 from Stable. (E.g. in case the following debugging didn't help
much.)

Yes.

Thanks for the debugging help!

Kinda, "local DEVICE" in the first line of
clevis_all_netbootable_devices() did help to make this error message
to go away, but it hung at boot nevertheless. So I debugged further.

This helped. "eno1" ended up being twice in $DEVICES (without "local
DEVICE" again). Very weird.

Indeed "local DEVICE" helped at this point to make the device to
appear only once and make the message go away. Not sure if this is the
best solution and has some unwanted side effects in other setups.
Especially because it still hang waiting for a passphrase and didn't
boot automatically

About 3 or 4 more hours of tracing the execution inside the initramfs
step by and reboot by reboot, I figured out that, at end, it fail at

        decrypted=$(echo -n "${jwe}" | clevis decrypt 2>/dev/null)

And due to the "2>/dev/null" we also don't see any potential error
message. Meh!

The suppressed error message is:

  Error communicating with the server!

And if I replace -s (silent) with -v in the curl command, I get "Could
not resolve host".

So Debian's initramfs doesn't seem to support DNS. And hence curl
couldn't resolve the hostname in the metadata. Doh.

But even adding the current /etc/resolv.conf via an initramfs-tools
hook didn't help.

The only thing what help was creating a custom /etc/hosts with the
IP and hostname of the host in the URL in the key metadata.

Then I removed that "local DEVICE" again and it still booted. So that
"multiple network interfaces" error message was a red herring. ARGH!

Probably not. dropbear-initramfs is probably in there, too, but I
don't count that one as non-standard.

# dpkg -l '*initramfs' | egrep '^ii' | cut -c-50
ii  clevis-initramfs             16-2
ii  cryptsetup-initramfs         2:2.3.7-1+deb11u1
ii  dropbear-initramfs           2020.81-3
# dpkg -S /usr/share/initramfs-tools/
xfsprogs, mdadm, udev, ntfs-3g, dropbear-initramfs, intel-microcode, clevis-initramfs, fuse3, cryptsetup-initramfs, initramfs-tools-core, lvm2, klibc-utils, busybox, kmod, dmsetup: /usr/share/initramfs-tools

There's a chance, yes. IIRC I occassionally have such issues in my
network. (The DHCP server is a Turris Omnia.)

But this wasn't an issue here.

So from my point of view this bug report can be solved by removing
the "2>/dev/null" from the line

        decrypted=$(echo -n "${jwe}" | clevis decrypt 2>/dev/null)

so that diagnostic network issues becomes much more easier.

		Regards, Axel

Hi Christoph,

Christoph Biedl wrote:

Well, yeah, depends what you do. If it's VPNs, SDNs and tunnels, it's
"never" MTU. ;-)

Well, yeah, wanted to get this off my desk before the soft freeze so
that you could potentially fix something if we find something to fix.

And getting clevis & tang work was on my TODO list anyways:

✅ Getting clevis and tang to work. ;-)

Yeah, about 4 hours. ;-)

Ok, will do. Need to reinstall some packages anyway as I happily
edited their inwards. or maybe I upgrade that box to Bookworm already
now. *g*

Indeed. I did a short "fgrep -w DEVICE
/usr/share/initramfs-tools/scripts/functions", but it seems that most
occurrences are in functions which are called after
clevis_all_netbootable_devices(). Then again, I'm sure
configure_network() ran twice, so maybe it comes from its first
invocation which started before clevis_all_netbootable_devices().

Probably. But also surely needs some testing in other situations, too.

Oh yeah.

A thought there: As I wasn't sure either how $(curl …) without -s has
impact on functionality, my debugging simply called curl a second time
in case of an error:

  if ! rep="$(curl -sfg -X POST -H "$ct" --data-binary @- "$url" <<< "$xfr")"; then
    echo "Error communicating with the server!" >&2
    # BEGIN DEBUGGING
    curl -vfg -X POST -H "$ct" --data-binary @- "$url" <<< "$xfr" || true
    cat /etc/resolv.conf
    ip a
    ip l
    ip r
    ping -c 1 my-tang-server.example.org
    # END DEBUGGING
    exit 1
  fi

But yes, this means that if then it works again, it will display
likely the secret. So maybe add some "> /dev/null" (without "2") or
so. Haven't tested the latter though.

		Regards, Axel

Es gibt eine Familienspende in Höhe von 1.850.000,00 USD von Cheng Charlie
Saephan. Bitte antworten Sie für weitere Informationen. Denken Sie daran,
Ihrer Familie und den Bedürftigen in Ihrer Umgebung Gutes zu tun.

Dies ist bereits der zweite Versuch, Sie zu erreichen. Bitte antworten Sie
für weitere Details.

Es gibt eine Familienspende in Höhe von 1.850.000,00 USD von Cheng Charlie
Saephan. Bitte antworten Sie für weitere Informationen. Denken Sie daran,
Ihrer Familie und den Bedürftigen in Ihrer Umgebung Gutes zu tun.

Dies ist bereits der zweite Versuch, Sie zu erreichen. Bitte antworten Sie
für weitere Details.