Using two Thunderbolt network interfaces as slaves in a bonding device in
mode 802.3ad (LACP) fails because the bonding driver cannot set the MAC
address on the thunderbolt_net interfaces.
The same setup works in mode active-backup.
kernel dmesg:
[ 25.922317] bond0: (slave thunderbolt0): The slave device
specified does not support setting the MAC address
[ 25.922328] bond0: (slave thunderbolt0): Error -95 calling
set_mac_address
[ 25.980235] bond0: (slave thunderbolt1): The slave device specified
does not support setting the MAC address
[ 25.980242] bond0: (slave thunderbolt1): Error -95 calling
set_mac_address
Steps to reproduce:
1. Create a bond with mode 802.3ad and add thunderbolt0 and thunderbolt1
as slaves.
2. Bring up the bond and slaves.
3. Observe that bonding fails to set the slave MAC addresses and logs the
errors above.
Expected result:
- bond0 and both Thunderbolt interfaces share bond0's MAC address.
- 802.3ad operates normally and the link comes up.
Actual result:
- dev_set_mac_address(thunderboltX, bond0_mac) fails with -EOPNOTSUPP.
- bonding reports that the slave does not support setting the MAC address
and cannot use the interfaces in 802.3ad mode.
Analysis:
- In drivers/net/thunderbolt/main.c the driver generates a locally
administered MAC from the Thunderbolt UUID and sets it with
eth_hw_addr_set().
- The net_device_ops for thunderbolt_net currently defines:
.ndo_open
.ndo_stop
.ndo_start_xmit
.ndo_get_stats64
but does not implement .ndo_set_mac_address.
- As a result, dev_set_mac_address() returns -EOPNOTSUPP, and bonding
treats the device as not supporting MAC address changes and logs the
error above.
From reading the driver, Thunderbolt networking appears to be purely
software/encapsulated Ethernet frames without a hardware MAC filter to
reprogram. It should therefore be possible to implement
ndo_set_mac_address using eth_mac_addr(), and optionally mark the device
with IFF_LIVE_ADDR_CHANGE so that MAC changes while the interface is up
are allowed.
Upstream (drivers/net/thunderbolt in Linus’ tree) also lacks
ndo_set_mac_address, so this would likely need to be fixed there as
well and then pulled into Debian.
If you’d like, I can test patches on this hardware.
Hi Ian, I'm defintively not fulent in the thunderbold specifics, but if the support is possible then you might want to bring that directly to upstream. You can contact the relevant people via: $ ./scripts/get_maintainer.pl drivers/net/thunderbolt Mika Westerberg <westeri@kernel.org> (maintainer:THUNDERBOLT NETWORK DRIVER) Yehezkel Bernat <YehezkelShB@gmail.com> (maintainer:THUNDERBOLT NETWORK DRIVER) Andrew Lunn <andrew+netdev@lunn.ch> (maintainer:NETWORKING DRIVERS) "David S. Miller" <davem@davemloft.net> (maintainer:NETWORKING DRIVERS) Eric Dumazet <edumazet@google.com> (maintainer:NETWORKING DRIVERS) Jakub Kicinski <kuba@kernel.org> (maintainer:NETWORKING DRIVERS) Paolo Abeni <pabeni@redhat.com> (maintainer:NETWORKING DRIVERS) netdev@vger.kernel.org (open list:THUNDERBOLT NETWORK DRIVER) linux-kernel@vger.kernel.org (open list) When you do fill the issue and request upstream, can you please keep this downstream bug in the loop so we are aware? Regards, Salvatore
Hi,
Using two Thunderbolt network interfaces as slaves in a bonding device
in mode 802.3ad (LACP) fails because the bonding driver cannot set the
MAC address on the thunderbolt_net interfaces. The same setup works in
mode active-backup.
Hardware: AMD Strix Halo (Framework connect to Sixunited AXB35 USB4 ports)
Kernel: 6.12.57 (also reproduced on 6.16.12 and 6.18~rc6)
Steps to reproduce:
1. Create a bond with mode 802.3ad and add thunderbolt0 and thunderbolt1
as slaves.
2. Bring up the bond and slaves.
3. Observe that bonding fails to set the slave MAC addresses and logs:
[ 25.922317] bond0: (slave thunderbolt0): The slave device
specified does not support setting the MAC address
[ 25.922328] bond0: (slave thunderbolt0): Error -95 calling
set_mac_address
[ 25.980235] bond0: (slave thunderbolt1): The slave device specified
does not support setting the MAC address
[ 25.980242] bond0: (slave thunderbolt1): Error -95 calling
set_mac_address
Expected result:
- bond0 and both Thunderbolt interfaces share bond0's MAC address.
- 802.3ad operates normally and the link comes up.
Actual result:
- dev_set_mac_address(thunderboltX, bond0_mac) fails with -EOPNOTSUPP.
- bonding reports that the slave does not support setting the MAC address
and cannot use the interfaces in 802.3ad mode.
From reading drivers/net/thunderbolt/main.c:
- thunderbolt_net generates a locally administered MAC from the
Thunderbolt UUID and sets it with eth_hw_addr_set().
- The net_device_ops for thunderbolt_net currently define:
.ndo_open
.ndo_stop
.ndo_start_xmit
.ndo_get_stats64
but do not implement .ndo_set_mac_address.
As a result, dev_set_mac_address() returns -EOPNOTSUPP and bonding treats
the device as not supporting MAC address changes.
A bit of research suggests it should be possible to implement
ndo_set_mac_address using
eth_mac_addr(), and, if appropriate, mark the device with
IFF_LIVE_ADDR_CHANGE so MAC changes while the interface is up are
allowed. I have a feeling there is a lot more to it;
There is a corresponding downstream Debian bug with additional
hardware details https://bugs.debian.org/1121032
Thanks,
Ian
Hi Ian, Okay "breaks" is probably too strong word here. It was never even supported :) Can you describe what are the actual commands you run so I can try to setup on my side and see how this could be implemented? Probably, I need to check this but first I need some way how to reproduce this :)
Hi,
Okay since the MAC address is not really being used in the USB4NET protocol
it should be fine to allow it to be changed.
The below allows me to change it using "ip link set" command. I wonder if
you could try it with the bonding case and see it that makes any
difference?
diff --git a/drivers/net/thunderbolt/main.c b/drivers/net/thunderbolt/main.c
index dcaa62377808..57b226afeb84 100644
--- a/drivers/net/thunderbolt/main.c
+++ b/drivers/net/thunderbolt/main.c
@@ -1261,6 +1261,7 @@ static const struct net_device_ops tbnet_netdev_ops = {
.ndo_open = tbnet_open,
.ndo_stop = tbnet_stop,
.ndo_start_xmit = tbnet_start_xmit,
+ .ndo_set_mac_address = eth_mac_addr,
.ndo_get_stats64 = tbnet_get_stats64,
};
@@ -1281,6 +1282,9 @@ static void tbnet_generate_mac(struct net_device *dev)
hash = jhash2((u32 *)xd->local_uuid, 4, hash);
addr[5] = hash & 0xff;
eth_hw_addr_set(dev, addr);
+
+ /* Allow changing it if needed */
+ dev->priv_flags |= IFF_LIVE_ADDR_CHANGE;
}
static int tbnet_probe(struct tb_service *svc, const struct tb_service_id *id)
On Fri, Nov 21, 2025 at 1:08 AM Mika Westerberg <mika.westerberg@linux.intel.com> wrote:
Agreed, let's say the "magic fades". I am guessing the same magic
that allows this 0x8086 component to appear out of thin air.
thunderbolt 0-2: new host found, vendor=0x8086 device=0x1
One side shown in netplan using a single yaml file (Ubuntu 24.04 server)
root@ai2:~# networkctl status bond0
● 3: bond0
Link File: /usr/lib/systemd/network/99-default.link
Network File: /run/systemd/network/10-netplan-bond0.network
State: routable (configured)
Online state: online
Type: bond
Kind: bond
Driver: bonding
Hardware Address: 02:92:d5:a7:f4:79
MTU: 1500 (min: 68, max: 65535)
QDisc: noqueue
IPv6 Address Generation Mode: eui64
Mode: active-backup
Miimon: 500ms
Updelay: 0
Downdelay: 0
Number of Queues (Tx/Rx): 16/16
Auto negotiation: no
Address: 10.10.13.2
fe80::92:d5ff:fea7:f479
Activation Policy: up
Required For Online: yes
DHCP6 Client DUID: DUID-EN/Vendor:0000ab11ccb509966215f387
Nov 21 16:10:03 ai2 systemd-networkd[720]: bond0: netdev ready
Nov 21 16:10:03 ai2 systemd-networkd[720]: bond0: Configuring with
/run/systemd/network/10-netplan-bond0.network.
Nov 21 16:10:03 ai2 systemd-networkd[720]: bond0: Link UP
Nov 21 16:10:08 ai2 systemd-networkd[720]: bond0: Gained carrier
Nov 21 16:10:09 ai2 systemd-networkd[720]: bond0: Gained IPv6LL
root@ai2:~# cat /etc/netplan/60-bonded-init.yaml
network:
version: 2
renderer: networkd
ethernets:
thunderbolt0:
dhcp4: false
thunderbolt1:
dhcp4: false
bonds:
bond0:
interfaces: [thunderbolt0, thunderbolt1]
dhcp4: false
addresses: [10.10.13.2/30]
parameters:
mode: active-backup
mii-monitor-interval: 500
The other side using a 3 file systemd-networkd variant (using on Debian 13)
ai4:/etc/systemd/network# networkctl status bond0
● 3: bond0
NetDev File: /etc/systemd/network/50-bond0.netdev
Link File: /usr/lib/systemd/network/99-default.link
Network File: /etc/systemd/network/53-bond0.network
State: routable (configured)
Online state: online
Type: bond
Kind: bond
Driver: bonding
Hardware Address: 02:0f:03:70:86:fb
MTU: 1500 (min: 68, max: 65535)
QDisc: noqueue
IPv6 Address Generation Mode: eui64
Mode: active-backup
Miimon: 500ms
Updelay: 0
Downdelay: 0
Number of Queues (Tx/Rx): 16/16
Auto negotiation: no
Address: 10.10.13.1
fe80::f:3ff:fe70:86fb
Activation Policy: up
Required For Online: yes
DHCPv6 Client DUID: DUID-EN/Vendor:0000ab112f49d10231f668bf
Nov 21 11:21:55 ai4 systemd-networkd[700]: bond0: netdev ready
Nov 21 11:21:55 ai4 systemd-networkd[700]: bond0: Configuring with
/etc/systemd/network/53-bond0.network.
Nov 21 11:21:55 ai4 systemd-networkd[700]: bond0: Link UP
Nov 21 11:22:01 ai4 systemd-networkd[700]: bond0: Gained carrier
Nov 21 11:22:02 ai4 systemd-networkd[700]: bond0: Gained IPv6LL
ai4:/etc/systemd/network# cat 50-bond0.netdev
# /etc/systemd/network/50-bond0.netdev
[NetDev]
Name=bond0
Kind=bond
[Bond]
MIIMonitorSec=0.5s
Mode=active-backup
FailOverMACPolicy=none
ai4:/etc/systemd/network# cat 52-thunderbolt-bond0-slaves.network
# /etc/systemd/network/52-thunderbolt-bond0-slaves.network
[Match]
Name=thunderbolt0 thunderbolt1
[Network]
Bond=bond0
ai4:/etc/systemd/network# cat 53-bond0.network
# /etc/systemd/network/53-bond0.network
[Match]
Name=bond0
[Network]
Address=10.10.13.1/30
Changing the mode to LACP/802.3ad then results in the observed mac
setting issues.
systemd-networkd/Debian Side:
ai4:/etc/systemd/network# cat 50-bond0.netdev
# /etc/systemd/network/50-bond0.netdev
[NetDev]
Name=bond0
Kind=bond
[Bond]
MIIMonitorSec=0.5s
Mode=802.3ad
TransmitHashPolicy=layer3+4
and the netplan/Ubuntu Side:
root@ai2:/etc/netplan# cat 60-bonded-init.yaml
network:
version: 2
renderer: networkd
ethernets:
thunderbolt0:
dhcp4: false
thunderbolt1:
dhcp4: false
bonds:
bond0:
interfaces: [thunderbolt0, thunderbolt1]
dhcp4: false
addresses: [10.10.13.2/30]
parameters:
mode: 802.3ad
transmit-hash-policy: layer3+4
mii-monitor-interval: 500
I typically reboot to apply the changes, to avoid some gaps in just
doing a netplan generate/apply or systemd-networkd restart, which do
not change the mode dynamically, as might be expected.
Sure, I can give this a shot this weekend
Hi Ian, Where you able to test the proposed change? Regards, Salvatore
Hi Mika,
Following up on the previous discussion, your patch enabling MAC address
changes allowed bonding to enslave thunderbolt_net devices, but 802.3ad
still could not form an aggregator because the driver does not report
link speed or duplex via ethtool. Bonding logs:
bond0: (slave thunderbolt0): failed to get link speed/duplex
Bonding (802.3ad) requires non-zero speed/duplex values for LACP port
key calculation.
The patch below adds a minimal get_link_ksettings() implementation and
registers ethtool_ops. It reports a fixed 10Gbps full-duplex link,
which is sufficient for LACP and seems consistent with ThunderboltIP
host-to-host bandwidth with the USB4v1/TB3 hardware I am using.
With this change, 802.3ad bonding comes up correctly on my USB4/TB
host-to-host setup. I also added link mode bitmaps, though they are not
strictly required for LACP/802.3ad.
Signed-off-by: Ian MacDonald <ian@netstatz.com>
---
drivers/net/thunderbolt/main.c | 29 +++++++++++++++++++++++++++++
1 file changed, 29 insertions(+)
diff --git a/drivers/net/thunderbolt/main.c b/drivers/net/thunderbolt/main.c
index 4f4694db6..b9e276693 100644
--- a/drivers/net/thunderbolt/main.c
+++ b/drivers/net/thunderbolt/main.c
@@ -15,6 +15,7 @@
#include <linux/jhash.h>
#include <linux/module.h>
#include <linux/etherdevice.h>
+#include <linux/ethtool.h>
#include <linux/rtnetlink.h>
#include <linux/sizes.h>
#include <linux/thunderbolt.h>
@@ -1257,6 +1258,28 @@ static void tbnet_get_stats64(struct net_device *dev,
stats->rx_missed_errors = net->stats.rx_missed_errors;
}
+static int tbnet_get_link_ksettings(struct net_device *dev,
+ struct ethtool_link_ksettings *cmd)
+{
+ /* ThunderboltIP is a software-only full-duplex network tunnel.
+ * We report fixed link settings to satisfy bonding (802.3ad)
+ * requirements for LACP port key calculation. Speed is set to
+ * 10Gbps as a conservative baseline.
+ */
+ ethtool_link_ksettings_zero_link_mode(cmd, supported);
+ ethtool_link_ksettings_add_link_mode(cmd, supported, 10000baseT_Full);
+
+ ethtool_link_ksettings_zero_link_mode(cmd, advertising);
+ ethtool_link_ksettings_add_link_mode(cmd, advertising, 10000baseT_Full);
+
+ cmd->base.speed = SPEED_10000;
+ cmd->base.duplex = DUPLEX_FULL;
+ cmd->base.autoneg = AUTONEG_DISABLE;
+ cmd->base.port = PORT_NONE;
+
+ return 0;
+}
+
static const struct net_device_ops tbnet_netdev_ops = {
.ndo_open = tbnet_open,
.ndo_stop = tbnet_stop,
@@ -1265,6 +1288,10 @@ static const struct net_device_ops tbnet_netdev_ops = {
.ndo_get_stats64 = tbnet_get_stats64,
};
+static const struct ethtool_ops tbnet_ethtool_ops = {
+ .get_link_ksettings = tbnet_get_link_ksettings,
+};
+
static void tbnet_generate_mac(struct net_device *dev)
{
const struct tbnet *net = netdev_priv(dev);
@@ -1315,6 +1342,7 @@ static int tbnet_probe(struct tb_service *svc,
const struct tb_service_id *id)
strcpy(dev->name, "thunderbolt%d");
dev->netdev_ops = &tbnet_netdev_ops;
+ dev->ethtool_ops = &tbnet_ethtool_ops;
/* ThunderboltIP takes advantage of TSO packets but instead of
* segmenting them we just split the packet into Thunderbolt
--
2.47.3
speeds within the expected range.
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID][Role] Interval Transfer Bitrate Retr
[ 5][TX-C] 0.00-10.07 sec 5.58 GBytes 4.76 Gbits/sec 0
sender
[ 5][TX-C] 0.00-10.07 sec 5.58 GBytes 4.76 Gbits/sec
receiver
[ 7][TX-C] 0.00-10.07 sec 5.58 GBytes 4.76 Gbits/sec 0
sender
[ 7][TX-C] 0.00-10.07 sec 5.58 GBytes 4.76 Gbits/sec
receiver
[ 9][TX-C] 0.00-10.07 sec 5.59 GBytes 4.77 Gbits/sec 0
sender
[ 9][TX-C] 0.00-10.07 sec 5.59 GBytes 4.77 Gbits/sec
receiver
[ 11][TX-C] 0.00-10.07 sec 5.59 GBytes 4.77 Gbits/sec 0
sender
[ 11][TX-C] 0.00-10.07 sec 5.59 GBytes 4.77 Gbits/sec
receiver
[SUM][TX-C] 0.00-10.07 sec 22.3 GBytes 19.1 Gbits/sec 0
sender
[SUM][TX-C] 0.00-10.07 sec 22.3 GBytes 19.1 Gbits/sec
receiver
[ 13][RX-C] 0.00-10.07 sec 3.72 GBytes 3.18 Gbits/sec 1
sender
[ 13][RX-C] 0.00-10.07 sec 3.72 GBytes 3.17 Gbits/sec
receiver
[ 15][RX-C] 0.00-10.07 sec 11.1 GBytes 9.50 Gbits/sec 4
sender
[ 15][RX-C] 0.00-10.07 sec 11.1 GBytes 9.50 Gbits/sec
receiver
[ 17][RX-C] 0.00-10.07 sec 3.72 GBytes 3.18 Gbits/sec 1
sender
[ 17][RX-C] 0.00-10.07 sec 3.72 GBytes 3.17 Gbits/sec
receiver
[ 19][RX-C] 0.00-10.07 sec 3.73 GBytes 3.18 Gbits/sec 1
sender
[ 19][RX-C] 0.00-10.07 sec 3.73 GBytes 3.18 Gbits/sec
receiver
[SUM][RX-C] 0.00-10.07 sec 22.3 GBytes 19.0 Gbits/sec 7
sender
[SUM][RX-C] 0.00-10.07 sec 22.3 GBytes 19.0 Gbits/sec
receiver
iperf Done.
ai2:~# iperf3 --bidir -c 10.10.13.1 -P 4 -t 10
ai2:~# networkctl status bond0
● 3: bond0
NetDev File: /etc/systemd/network/50-bond0.netdev
Link File: /usr/lib/systemd/network/99-default.link
Network File: /etc/systemd/network/53-bond0.network
State: routable (configured)
Online state: online
Type: bond
Kind: bond
Driver: bonding
Hardware Address: 82:36:12:ad:a1:c0
MTU: 1500 (min: 68, max: 65535)
QDisc: noqueue
IPv6 Address Generation Mode: eui64
Mode: 802.3ad
Miimon: 500ms
Updelay: 0
Downdelay: 0
Number of Queues (Tx/Rx): 16/16
Auto negotiation: no
Speed: 20Gbps
Duplex: full
Address: 10.10.13.2
fe80::8036:12ff:fead:a1c0
Activation Policy: up
Required For Online: yes
DHCPv6 Client DUID: DUID-EN/Vendor:0000ab111c6e5c59896f0172
Nov 26 22:46:11 ai2 systemd-networkd[641]: bond0: netdev ready
Nov 26 22:46:11 ai2 systemd-networkd[641]: bond0: Configuring with
/etc/systemd/network/53-bond0.network.
Nov 26 22:46:11 ai2 systemd-networkd[641]: bond0: Link UP
Nov 26 17:46:52 ai2 systemd-networkd[641]: bond0: Gained carrier
Nov 26 17:46:53 ai2 systemd-networkd[641]: bond0: Gained IPv6LL
ai2:~# ethtool thunderbolt0
Settings for thunderbolt0:
Supported ports: [ ]
Supported link modes: 10000baseT/Full
Supported pause frame use: No
Supports auto-negotiation: No
Supported FEC modes: Not reported
Advertised link modes: 10000baseT/Full
Advertised pause frame use: No
Advertised auto-negotiation: No
Advertised FEC modes: Not reported
Speed: 10000Mb/s
Duplex: Full
Auto-negotiation: off
Port: None
PHYAD: 0
Transceiver: internal
Hi, Okay thanks for checking. I see. Yeah for speed we need to do something more than use the hard-coded values because it varies depending on whether we have lane bonding (not the same bonding you are doing) enabled and also how the link trained. Let me come up with a patch based on yours that does that.
ethtool header: https://elixir.bootlin.com/linux/v6.17.9/source/include/uapi/linux/ethtool.h#L2177 However, ethtool might report a warning it does not know how to convert to a nice looking number. So if you only have a small number of possible speeds, please consider extending the list in ethtool, and phy_speed_to_str(). Are link modes needed by bonding? You technically don't have a baseT link, its not four twisted pairs, RJ45 connectors etc. Also, since you set autoneg to disabled, you are not advertising anything. If the bond drive does not care, i would leave supported and advertising empty. Andrew