Fabre

#927397 u-boot: Olimex Lime2 Rev.F-G2: severe packet loss w/ gigabit ethernet #927397

Package:: src:u-boot

Source:: u-boot

Submitter:: Sunil Mohan Adapa

Date:: 2022-06-09 10:36:06 UTC

Severity:: important

#927397#5

Date:: 2019-04-18 23:43:37 UTC

From:

To:

Dear Maintainer,

Olimex team working with FreedomBox team have noticed a significant slowdown in
Gigabit Ethernet when transmitting. Original report from Olimex team is
attached. This is applicable for hardware Rev.G2 of the A20 OLinuXino Lime2
board.

Olimex team have sent a patch that fixes the issue for Rev.G2 (attached).
Multiple team members from FreedomBox have confirmed that the patch fixes
transmit performance issue for Rev.G2 and yields good transmit/receive
performance. However this patch makes the situation worse for Rev.C (my test
results attached). I believe this issue is separate from #845128 (will post
more information there) which already causes *receive* performance to be bad on
Rev.C.

=====================================
Report from Olimex team (with Rev.G2)
=====================================

Our test shows that speed in TX side is very slow and unstable. The problem is
in the wrong clock delay settings. We corrected this settings in our image and
recommend you to change the TX delay too. You have to change value of
CCM_GMAC_CTRL_TX_CLK_DELAY register on address 0x01c20164 from 0x00000006 to
0x00001006. This should be changed in the u-boot sources and the image should
be  compiled again.

- ------------------------------------------------------------------------------
Tests with original image. Unchaged CCM_GMAC_CTRL_TX_CLK_DELAY value(00000006)
- ------------------------------------------------------------------------------

=> md.l 0x01c20164 1
01c20164: 00000006


admin@freedombox:~$ iperf3 -c 192.168.0.60 -p 2001
Connecting to host 192.168.0.60, port 2001
[  5] local 192.168.0.135 port 57448 connected to 192.168.0.60 port 2001
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   375 KBytes  3.07 Mbits/sec   32   4.24 KBytes
[  5]   1.00-2.00   sec  1.67 MBytes  14.0 Mbits/sec  147   9.90 KBytes
[  5]   2.00-3.00   sec  4.49 MBytes  37.6 Mbits/sec  261   2.83 KBytes
[  5]   3.00-4.00   sec   669 KBytes  5.48 Mbits/sec   61   5.66 KBytes
[  5]   4.00-5.00   sec  2.78 MBytes  23.3 Mbits/sec  160   1.41 KBytes
[  5]   5.00-6.00   sec   829 KBytes  6.79 Mbits/sec   88   1.41 KBytes
[  5]   6.00-7.00   sec  1.47 MBytes  12.4 Mbits/sec   94   2.83 KBytes
[  5]   7.00-8.00   sec  1.55 MBytes  13.0 Mbits/sec   98   1.41 KBytes
[  5]   8.00-9.00   sec  2.47 MBytes  20.8 Mbits/sec  169   1.41 KBytes
[  5]   9.00-10.00  sec   700 KBytes  5.73 Mbits/sec   44   2.83 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  16.9 MBytes  14.2 Mbits/sec  1154             sender
[  5]   0.00-10.00  sec  16.8 MBytes  14.1 Mbits/sec                  receiver

iperf Done.
admin@freedombox:~$ iperf3 -c 192.168.0.60 -p 2001 -R
Connecting to host 192.168.0.60, port 2001
Reverse mode, remote host 192.168.0.60 is sending
[  5] local 192.168.0.135 port 57452 connected to 192.168.0.60 port 2001
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  97.1 MBytes   814 Mbits/sec
[  5]   1.00-2.00   sec  98.2 MBytes   824 Mbits/sec
[  5]   2.00-3.00   sec  98.5 MBytes   826 Mbits/sec
[  5]   3.00-4.00   sec  48.6 MBytes   408 Mbits/sec
[  5]   4.00-5.00   sec  98.2 MBytes   824 Mbits/sec
[  5]   5.00-6.00   sec  97.9 MBytes   821 Mbits/sec
[  5]   6.00-7.00   sec  98.2 MBytes   824 Mbits/sec
[  5]   7.00-8.00   sec  97.7 MBytes   820 Mbits/sec
[  5]   8.00-9.00   sec  98.4 MBytes   826 Mbits/sec
[  5]   9.00-10.00  sec  27.5 MBytes   231 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   861 MBytes   722 Mbits/sec   58             sender
[  5]   0.00-10.00  sec   860 MBytes   722 Mbits/sec                  receiver

iperf Done.


- -------------------------------------------------------------------------
Test with CCM_GMAC_CTRL_TX_CLK_DELAY value set to 00001006
- -------------------------------------------------------------------------

=> md.l 0x01c20164 1
01c20164: 00001006                               ....


admin@freedombox:~$ iperf3 -c 192.168.0.60 -p 2001
Connecting to host 192.168.0.60, port 2001
[  5] local 192.168.0.135 port 53002 connected to 192.168.0.60 port 2001
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  47.5 MBytes   398 Mbits/sec    0    153 KBytes
[  5]   1.00-2.01   sec  53.8 MBytes   448 Mbits/sec    0    206 KBytes
[  5]   2.01-3.00   sec  32.8 MBytes   277 Mbits/sec    1    235 KBytes
[  5]   3.00-4.00   sec  67.5 MBytes   567 Mbits/sec    0    235 KBytes
[  5]   4.00-5.00   sec  56.2 MBytes   471 Mbits/sec    0    235 KBytes
[  5]   5.00-6.02   sec  48.8 MBytes   400 Mbits/sec    0    235 KBytes
[  5]   6.02-7.01   sec  47.5 MBytes   402 Mbits/sec    0    235 KBytes
[  5]   7.01-8.02   sec  48.8 MBytes   406 Mbits/sec    0    235 KBytes
[  5]   8.02-9.00   sec  60.5 MBytes   516 Mbits/sec    0    267 KBytes
[  5]   9.00-10.01  sec  47.5 MBytes   396 Mbits/sec    0    267 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.01  sec   511 MBytes   428 Mbits/sec    1             sender
[  5]   0.00-10.01  sec   511 MBytes   428 Mbits/sec                  receiver

iperf Done.
admin@freedombox:~$ iperf3 -c 192.168.0.60 -p 2001 -R
Connecting to host 192.168.0.60, port 2001
Reverse mode, remote host 192.168.0.60 is sending
[  5] local 192.168.0.135 port 53006 connected to 192.168.0.60 port 2001
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  46.1 MBytes   387 Mbits/sec
[  5]   1.00-2.00   sec  97.2 MBytes   816 Mbits/sec
[  5]   2.00-3.00   sec  97.2 MBytes   815 Mbits/sec
[  5]   3.00-4.00   sec  97.2 MBytes   815 Mbits/sec
[  5]   4.00-5.00   sec  97.4 MBytes   816 Mbits/sec
[  5]   5.00-6.00   sec  97.1 MBytes   815 Mbits/sec
[  5]   6.00-7.00   sec  97.3 MBytes   817 Mbits/sec
[  5]   7.00-8.00   sec  97.7 MBytes   820 Mbits/sec
[  5]   8.00-9.00   sec  97.5 MBytes   818 Mbits/sec
[  5]   9.00-10.00  sec  97.6 MBytes   818 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   923 MBytes   774 Mbits/sec   48             sender
[  5]   0.00-10.00  sec   922 MBytes   774 Mbits/sec                  receiver

===================================
Test results without patch on Rev.C
===================================
root@freedombox:~# iperf3 -c 10.42.1.1 -R
Connecting to host 10.42.1.1, port 5201
Reverse mode, remote host 10.42.1.1 is sending
[  5] local 10.42.1.139 port 48228 connected to 10.42.1.1 port 5201
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec   281 KBytes  2.30 Mbits/sec
[  5]   1.00-2.00   sec  60.8 KBytes   498 Kbits/sec
[  5]   2.00-3.00   sec  0.00 Bytes  0.00 bits/sec
[  5]   3.00-4.00   sec   158 KBytes  1.30 Mbits/sec
[  5]   4.00-5.00   sec   235 KBytes  1.92 Mbits/sec
[  5]   5.00-6.00   sec   110 KBytes   903 Kbits/sec
[  5]   6.00-7.00   sec   228 KBytes  1.86 Mbits/sec
[  5]   7.00-8.00   sec   109 KBytes   892 Kbits/sec
[  5]   8.00-9.00   sec  0.00 Bytes  0.00 bits/sec
[  5]   9.00-10.00  sec   112 KBytes   916 Kbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.04  sec  1.38 MBytes  1.15 Mbits/sec  188             sender
[  5]   0.00-10.00  sec  1.26 MBytes  1.06 Mbits/sec                  receiver

iperf Done.
root@freedombox:~# iperf3 -c 10.42.1.1
Connecting to host 10.42.1.1, port 5201
[  5] local 10.42.1.139 port 48232 connected to 10.42.1.1 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.01   sec  65.8 MBytes   545 Mbits/sec    0    468 KBytes
[  5]   1.01-2.01   sec  67.5 MBytes   567 Mbits/sec    0    468 KBytes
[  5]   2.01-3.01   sec  66.5 MBytes   556 Mbits/sec    0    673 KBytes
[  5]   3.01-4.00   sec  67.2 MBytes   570 Mbits/sec    0    841 KBytes
[  5]   4.00-5.01   sec  53.8 MBytes   450 Mbits/sec    0    841 KBytes
[  5]   5.01-6.01   sec  65.0 MBytes   545 Mbits/sec    0    841 KBytes
[  5]   6.01-7.02   sec  46.2 MBytes   384 Mbits/sec    0    841 KBytes
[  5]   7.02-8.02   sec  66.2 MBytes   556 Mbits/sec    0    884 KBytes
[  5]   8.02-9.00   sec  68.6 MBytes   583 Mbits/sec    0    884 KBytes
[  5]   9.00-10.01  sec  63.1 MBytes   525 Mbits/sec    0   1.09 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.01  sec   630 MBytes   528 Mbits/sec    0             sender
[  5]   0.00-10.05  sec   630 MBytes   526 Mbits/sec                  receiver

iperf Done.
root@freedombox:~# mii-tool eth0 -A 100BaseTx-FD
restarting autonegotiation...
root@freedombox:~# iperf3 -c 10.42.1.1
Connecting to host 10.42.1.1, port 5201
[  5] local 10.42.1.139 port 48236 connected to 10.42.1.1 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  11.5 MBytes  96.0 Mbits/sec    0    119 KBytes
[  5]   1.00-2.00   sec  11.2 MBytes  94.4 Mbits/sec    0    124 KBytes
[  5]   2.00-3.00   sec  11.1 MBytes  92.7 Mbits/sec    0    139 KBytes
[  5]   3.00-4.00   sec  11.2 MBytes  94.3 Mbits/sec    0    139 KBytes
[  5]   4.00-5.00   sec  11.2 MBytes  94.4 Mbits/sec    0    139 KBytes
[  5]   5.00-6.00   sec  11.2 MBytes  93.9 Mbits/sec    0    139 KBytes
[  5]   6.00-7.00   sec  11.3 MBytes  94.9 Mbits/sec    0    139 KBytes
[  5]   7.00-8.00   sec  11.2 MBytes  93.8 Mbits/sec    0    139 KBytes
[  5]   8.00-9.00   sec  11.2 MBytes  93.8 Mbits/sec    0    139 KBytes
[  5]   9.00-10.00  sec  11.2 MBytes  93.8 Mbits/sec    0    139 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   112 MBytes  94.2 Mbits/sec    0             sender
[  5]   0.00-10.04  sec   112 MBytes  93.6 Mbits/sec                  receiver

iperf Done.
root@freedombox:~# iperf3 -c 10.42.1.1 -R
Connecting to host 10.42.1.1, port 5201
Reverse mode, remote host 10.42.1.1 is sending
[  5] local 10.42.1.139 port 48240 connected to 10.42.1.1 port 5201
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  11.2 MBytes  94.3 Mbits/sec
[  5]   1.00-2.00   sec  11.2 MBytes  93.7 Mbits/sec
[  5]   2.00-3.00   sec  11.2 MBytes  93.7 Mbits/sec
[  5]   3.00-4.00   sec  11.2 MBytes  93.7 Mbits/sec
[  5]   4.00-5.00   sec  11.2 MBytes  93.7 Mbits/sec
[  5]   5.00-6.00   sec  11.2 MBytes  93.7 Mbits/sec
[  5]   6.00-7.00   sec  11.2 MBytes  93.7 Mbits/sec
[  5]   7.00-8.00   sec  11.2 MBytes  93.7 Mbits/sec
[  5]   8.00-9.00   sec  11.2 MBytes  93.7 Mbits/sec
[  5]   9.00-10.00  sec  11.2 MBytes  93.7 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.04  sec   113 MBytes  94.6 Mbits/sec    0             sender
[  5]   0.00-10.00  sec   112 MBytes  93.7 Mbits/sec                  receiver

iperf Done.

================================
Test results with patch on Rev.C
================================

root@freedombox:~# iperf3 -c 10.42.1.1
Connecting to host 10.42.1.1, port 5201
[  5] local 10.42.1.139 port 46460 connected to 10.42.1.1 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  63.6 KBytes   521 Kbits/sec   19   1.41 KBytes
[  5]   1.00-2.00   sec  0.00 Bytes  0.00 bits/sec    1   1.41 KBytes
[  5]   2.00-3.00   sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes
[  5]   3.00-4.00   sec  0.00 Bytes  0.00 bits/sec    8   2.83 KBytes
[  5]   4.00-5.00   sec  0.00 Bytes  0.00 bits/sec    2   1.41 KBytes
[  5]   5.00-6.00   sec  0.00 Bytes  0.00 bits/sec    1   1.41 KBytes
[  5]   6.00-7.00   sec  0.00 Bytes  0.00 bits/sec    3   1.41 KBytes
[  5]   7.00-8.00   sec  43.8 KBytes   359 Kbits/sec    3   1.41 KBytes
[  5]   8.00-9.00   sec  0.00 Bytes  0.00 bits/sec    5   1.41 KBytes
[  5]   9.00-10.00  sec  0.00 Bytes  0.00 bits/sec   12   2.83 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   107 KBytes  88.0 Kbits/sec   54             sender
[  5]   0.00-10.04  sec  74.9 KBytes  61.1 Kbits/sec                  receiver

iperf Done.
root@freedombox:~# iperf3 -c 10.42.1.1 -R
Connecting to host 10.42.1.1, port 5201
Reverse mode, remote host 10.42.1.1 is sending
[  5] local 10.42.1.139 port 46464 connected to 10.42.1.1 port 5201
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec   107 KBytes   880 Kbits/sec
[  5]   1.00-2.00   sec  1.41 KBytes  11.6 Kbits/sec
[  5]   2.00-3.00   sec  29.7 KBytes   243 Kbits/sec
[  5]   3.00-4.00   sec  7.07 KBytes  57.9 Kbits/sec
[  5]   4.00-5.00   sec  1.41 KBytes  11.6 Kbits/sec
[  5]   5.00-6.00   sec  0.00 Bytes  0.00 bits/sec
[  5]   6.00-7.04   sec  11.3 KBytes  88.9 Kbits/sec
[  5]   7.04-8.00   sec  5.66 KBytes  48.4 Kbits/sec
[  5]   8.00-9.00   sec  4.24 KBytes  34.7 Kbits/sec
[  5]   9.00-10.00  sec  4.24 KBytes  34.7 Kbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.01  sec   255 KBytes   208 Kbits/sec   54             sender
[  5]   0.00-10.00  sec   173 KBytes   141 Kbits/sec                  receiver

iperf Done.
root@freedombox:~# mii-tool eth0 -A 100BaseTx-FD
restarting autonegotiation...
root@freedombox:~# iperf3 -c 10.42.1.1 -R
Connecting to host 10.42.1.1, port 5201
Reverse mode, remote host 10.42.1.1 is sending
[  5] local 10.42.1.139 port 46472 connected to 10.42.1.1 port 5201
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  11.2 MBytes  94.3 Mbits/sec
[  5]   1.00-2.00   sec  11.2 MBytes  93.7 Mbits/sec
[  5]   2.00-3.00   sec  11.2 MBytes  93.7 Mbits/sec
[  5]   3.00-4.00   sec  11.2 MBytes  93.7 Mbits/sec
[  5]   4.00-5.00   sec  11.2 MBytes  93.7 Mbits/sec
[  5]   5.00-6.00   sec  11.2 MBytes  93.7 Mbits/sec
[  5]   6.00-7.00   sec  5.82 MBytes  48.8 Mbits/sec
[  5]   7.00-8.00   sec  11.2 MBytes  93.5 Mbits/sec
[  5]   8.00-9.00   sec  11.1 MBytes  93.3 Mbits/sec
[  5]   9.00-10.00  sec  11.2 MBytes  93.7 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.04  sec   107 MBytes  89.7 Mbits/sec    1             sender
[  5]   0.00-10.00  sec   106 MBytes  89.2 Mbits/sec                  receiver

iperf Done.
root@freedombox:~# iperf3 -c 10.42.1.1
Connecting to host 10.42.1.1, port 5201
[  5] local 10.42.1.139 port 46476 connected to 10.42.1.1 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  11.4 MBytes  95.9 Mbits/sec    0    116 KBytes
[  5]   1.00-2.00   sec  11.2 MBytes  94.4 Mbits/sec    0    127 KBytes
[  5]   2.00-3.00   sec  11.4 MBytes  95.4 Mbits/sec    0    134 KBytes
[  5]   3.00-4.00   sec  11.2 MBytes  93.8 Mbits/sec    0    134 KBytes
[  5]   4.00-5.00   sec  5.97 MBytes  50.0 Mbits/sec    0    134 KBytes
[  5]   5.00-6.00   sec  10.8 MBytes  90.7 Mbits/sec    0    140 KBytes
[  5]   6.00-7.00   sec  11.2 MBytes  93.8 Mbits/sec    0    140 KBytes
[  5]   7.00-8.00   sec  11.4 MBytes  95.4 Mbits/sec    0    140 KBytes
[  5]   8.00-9.00   sec  11.2 MBytes  93.8 Mbits/sec    0    140 KBytes
[  5]   9.00-10.00  sec  11.2 MBytes  94.3 Mbits/sec    0    140 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   107 MBytes  89.8 Mbits/sec    0             sender
[  5]   0.00-10.04  sec   107 MBytes  89.1 Mbits/sec                  receiver

iperf Done.



- -- System Information:
Debian Release: buster/sid
  APT prefers testing
  APT policy: (500, 'testing')
Architecture: amd64 (x86_64)

Kernel: Linux 4.19.0-4-amd64 (SMP w/4 CPU cores)
Kernel taint flags: TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE
Locale: LANG=en_IN.UTF-8, LC_CTYPE=en_IN.UTF-8 (charmap=UTF-8), LANGUAGE=en_IN.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled
-----BEGIN PGP SIGNATURE-----

iQJFBAEBCgAvFiEE5xPDY9ZyWnWupXSBQ+oc/wqnxfIFAly5C6YRHHN1bmlsQG1l
ZGhhcy5vcmcACgkQQ+oc/wqnxfL8XA//ZUafL2tt7IN813ild+134HPKYrygL0LE
SCKEJCMSf62rzAyu0MX8+CKLFKY39j9W21SuZ5mdBVFEXY2dOxvSd+JRwqldJZx7
uTjycsAT6pu0tAwgGgAetDXWTiW0WGDuoi+x/g0Pnsa9GQRd77hNRZ70b9rwGzSI
Hi/YAAt6MGvKb+SgyxQ+xzVa9wVKnQVs/7/aM7q2GHF7VxAtpKSO/FEEsKPKa0tn
YIfQNq7SuiCAIsKDGE6sXN1Vf/uKdxz03G9rdNwMSMAQZpvOwx/e0Tz3jbZWszz9
xXmCl2ZAli0p12sKGzR5J8yh5UGBFaeVx58BNbfrOS1sDdYpuEG0Ynp6F5bliqHR
0vjK2GXyL2rQAtEzm5iuERvm4+5ukf6X7hqbkXfoDcuC5edsJiXoQ3u0gc4r4fbu
s8proHISS0cRTGokIl2MW7K++Q+617kxG5FNf0x7Osw7NJ7nY769OqoPJ9MTSQg+
JjjdNJ5H4BahN4vDTz1rAI+NledP5kWaxL1qGUYfWfRtooG27aaU3NS5cWamizbo
BmJgaVUYvFX91jtrJoIiAUM/o60AWxNIR8VbZigegRvihcuBgD68Q4Xk6l2C0Oj5
2yIKQVPCAfac4ahVnqKkEeJWaD8MFZ5JzFkHWREU9h218xpf7k5Dopwjy2HMgvJ7
odVoPsw94C4=
=uqyK
-----END PGP SIGNATURE-----

#927397#10

Date:: 2019-04-21 10:27:59 UTC

From:

To:

Hi Sunil,

Do I understand you correctly that the attached patch is what Olimex
propose but that you do *not* recommended to use it as-is because it
badly affects older boards?

Were your Lime2 boards connected with a cross-over cable or via a switch
during those tests?

I am no expert on these things, but did some digging...:

Older Lime2 use RTL8211CL-GR and Rev.G onwards use RTL8211E-VB-CG1,
according to
https://github.com/OLIMEX/OLINUXINO/tree/master/HARDWARE/A20-OLinuXino-LIME2

Lime2 RTL8211C has a buggy PLL, and u-boot since 2016.11+dfsg1-2 force
RTL8211C (but *not* RTL8211E) to run as master when in gigabit mode,
according to u-boot commit cebf3f5

Upstream discussion and tests when introducing CONFIG_GMAC_TX_DELAY:
https://lists.denx.de/pipermail/u-boot/2016-March/248498.html

Above upstream tests most likely used only the older RTL8211C PHY, as
rev.G was seemingly introduced mid 2016 at earliest, according to
https://olimex.wordpress.com/2016/12/08/a20-olinuxino-lime2-now-with-pcb-revision-g/
and
https://olimex.wordpress.com/2016/05/04/lime2-get-better-now-with-emmc-flash-a20-olinuxino-lime2-emmc/
and (vaguely)
https://github.com/OLIMEX/OLINUXINO/blob/master/HARDWARE/A20-OLinuXino-LIME2/hardware_revision_changes_log.txt#L61


 - Jonas

#927397#15

Date:: 2019-04-25 12:58:13 UTC

From:

To:

It seems mainline u-boot bootstrao all lime2 devices equally, whereas
Olimex fork of u-boot treats "newer than G and "newer than E" specially:
https://github.com/OLIMEX/u-boot/commit/6c32a3c9d31432884751966fcb0f15b1fd930446

  * Realtek rev.C PHY (board rev.C) get no tweak (but see bug#845128)
  * Realtek rev.E PHY (board rev.G,G1,G2) get TX_DELAY=2
  * Micrel PHY (board rev.H,K) get TX_DELAY=4

...as pointed out on irc up until here:
https://freenode.irclog.whitequark.org/linux-sunxi/2019-04-25#24483567


 - Jonas

#927397#20

Date:: 2019-04-25 18:55:30 UTC

From:

To:

The following is the patch Olimex has applied on u-boot for the images
that they build. It is meant to work for all hardware revisions of Lime2.

https://github.com/OLIMEX/OLINUXINO/blob/master/SOFTWARE/A20/A20-build-3.4.103-release-7/a20-phy_1000_100-dram.patch

It does not seem suitable for non-Lime2 boards and may need changes to
before it can be submitted upstream (or Debian).

Thanks,

#927397#25

Date:: 2019-04-25 19:30:03 UTC

From:

To:

The report is in the inline section "Report from Olimex team (with
Rev.G2)". I used term 'attached' loosely.

Olimex has kindly provided us the patch so that we can create a fully
working u-boot build for Lime2 Rev.G2 board. They did not imply that the
patch was suitable for other boards as well.

Lime2 was connected to a laptop via a cross-over cable (actually regular
cable but the hardware actually detects cross-over setup and
automatically swaps TX/RX).

[...]

#927397#30

Date:: 2019-04-26 07:54:37 UTC

From:

To:

Quoting Sunil Mohan Adapa (2019-04-25 21:30:03)
down to and including the iperf tests on ip 192.168.0.60, and then you
_appended_ tests of your own for a rev.C board on ip 10.42.1.1.

Understood (now).  Thanks.

Thanks.  Makes good sense to me now (sorry for being dense).

So a summary of the tests is this:

Board, cable mode, patchset \ Measured speed in Mbit/sec
rev.G2 1Gbit mode pristine:    14 & 722
rev.G2 1Gbit mode TX_DELAY=4: 428 & 774
rev.C 1Gbit pristine:           1 & 526
rev.C 100Mbit pristine:        94 &  94
rev.C 1Gbit TX_DELAY=4:        61 & 141
rev.C 100Mbit TX_DELAY=4:      89 &  89

Would be good to have tests explicitly in master vs. slave mode.

Would be good to have tests with TX_DELAY=2 and TX_DELAY=3 (the former
seems to be what Olimex set themselves for rev.G, and the latter is what
some u-boot hacker has noted as working on linux-sunxi wiki).

Would be good to have tests for rev.H and/or rev. K board (to understand
if changes cause regressions on that board - yes I am aware that some
FreedomBox reports indicate that board being broken too but we
desperately need detailed knowledge on that filed in the Debian BTS!).


 - Jonas

#927397#35

Date:: 2019-04-26 07:59:48 UTC

From:

To:

Quoting Jonas Smedegaard (2019-04-26 09:54:37)

Seems master/slave can at least be probed (and possibly forced) in linux
with ethtool, and in u-boot with mii

 - Jonas

#927397#40

Date:: 2019-04-27 09:46:44 UTC

From:

To:

Hi all,

I tried to reproduce the results for my rev.G2 board (2017), but could
not reproduce the bad performance.  My results are:

    ansible@blackbox:~$ iperf3 -c 192.168.122.1
    Connecting to host 192.168.122.1, port 5201
    [  5] local 192.168.122.223 port 60618 connected to 192.168.122.1 port 5201
    [ ID] Interval           Transfer     Bitrate         Retr  Cwnd
    [  5]   0.00-1.00   sec  70.0 MBytes   587 Mbits/sec    0    752 KBytes
    [  5]   1.00-2.00   sec  68.9 MBytes   578 Mbits/sec    0    923 KBytes
    [  5]   2.00-3.00   sec  67.4 MBytes   566 Mbits/sec    0   1012 KBytes
    [  5]   3.00-4.00   sec  65.7 MBytes   550 Mbits/sec    0   1.15 MBytes
    [  5]   4.00-5.00   sec  65.0 MBytes   546 Mbits/sec    0   1.21 MBytes
    [  5]   5.00-6.00   sec  68.8 MBytes   577 Mbits/sec   42    904 KBytes
    [  5]   6.00-7.00   sec  67.5 MBytes   564 Mbits/sec    0    929 KBytes
    [  5]   7.00-8.01   sec  52.5 MBytes   437 Mbits/sec    0    929 KBytes
    [  5]   8.01-9.00   sec  55.0 MBytes   465 Mbits/sec    0   1.08 MBytes
    [  5]   9.00-10.01  sec  56.2 MBytes   468 Mbits/sec   18    803 KBytes
    - - - - - - - - - - - - - - - - - - - - - - - - -
    [ ID] Interval           Transfer     Bitrate         Retr
    [  5]   0.00-10.01  sec   637 MBytes   534 Mbits/sec   60             sender
    [  5]   0.00-10.05  sec   637 MBytes   532 Mbits/sec                  receiver

    iperf Done.
    ansible@blackbox:~$ iperf3 -c 192.168.122.1 -R
    Connecting to host 192.168.122.1, port 5201
    Reverse mode, remote host 192.168.122.1 is sending
    [  5] local 192.168.122.223 port 60622 connected to 192.168.122.1 port 5201
    [ ID] Interval           Transfer     Bitrate
    [  5]   0.00-1.00   sec  94.0 MBytes   788 Mbits/sec
    [  5]   1.00-2.00   sec   106 MBytes   888 Mbits/sec
    [  5]   2.00-3.00   sec   108 MBytes   906 Mbits/sec
    [  5]   3.00-4.00   sec   111 MBytes   933 Mbits/sec
    [  5]   4.00-5.00   sec   111 MBytes   933 Mbits/sec
    [  5]   5.00-6.00   sec   111 MBytes   933 Mbits/sec
    [  5]   6.00-7.00   sec   111 MBytes   932 Mbits/sec
    [  5]   7.00-8.00   sec   111 MBytes   933 Mbits/sec
    [  5]   8.00-9.00   sec   108 MBytes   908 Mbits/sec
    [  5]   9.00-10.00  sec   109 MBytes   917 Mbits/sec
    - - - - - - - - - - - - - - - - - - - - - - - - -
    [ ID] Interval           Transfer     Bitrate         Retr
    [  5]   0.00-10.04  sec  1.06 GBytes   906 Mbits/sec  179             sender
    [  5]   0.00-10.00  sec  1.06 GBytes   907 Mbits/sec                  receiver

    iperf Done.

U-boot is the version from experimental, 2019.04+dfsg-2.

Regards,

  Andi

#927397#45

Date:: 2019-04-28 04:31:21 UTC

From:

To:

Oops! Forget to mention an important detail. Only about 20% of the
Rev.G2 boards that Olimex tested (out of a large number) were facing
this issue. We were also not able to reproduce the problem on one of the
Rev.G2 boards. However, Olimex assures us that the problem is real. They
mentioned that tolerances ranges of various hardware components could be
the reason for only some boards facing this issue and not all. Setting
TX_DELAY=4 works on 100% of all Rev. G2  boards.

When releasing FreedomBox Pioneer Edition this week, we had to
unfortunately ship with a patched version of u-boot with TX_DELAY=4.

#927397#50

Date:: 2019-11-05 15:14:49 UTC

From:

To:

Here's what I know (the "should work" entries need confirmation):

Lime2 rev.C
-----------

* Uses Realtek rev.C PHY
* Works fine in 1Gbit mode with Debian stable U-boot
* Works fine in 1Gbit mode with Debian stable kernel
* Possibly still sold as some no-eMMC no-flash options

Lime2 rev.G
-----------

* Uses Realtek rev.E PHY
* Sold as FreedomBox Edition
* Possibly still sold as some no-flash options
* Works in 100Mbit mode with Debian-stable kernel and u-boot
* Works in 1Gbit mode with Debian-stable kernel
  and custom U-boot patched to set TX_DELAY=4

Lime rev.H
----------

* Uses Micrel PHY
* Sold (at least) as flash options
* Works in 10Mbit mode with Debian stable kernel and u-boot
* Should work in 1Gbit mode with Debian-backports kernel v5.2
* Should work in 1Gbit mode with custom kernel
  patched to to set CONFIG_MICREL_PHY=y (already done with Debian kernels)
  and to include https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=3aed3e2
  (applied upstream since v5.2)


With systemd-networkd, all lime2 boards work with this added as file
/etc/systemd/network/90-ethernet.link (and rev.G boards work also
with 100baset entries uncommented):

[Match]
Driver=st_gmac

[Link]
NamePolicy=keep kernel database onboard slot path
MACAddressPolicy=persistent

Advertise=10baset-half
Advertise=10baset-full
#Advertise=100baset-half
#Advertise=100baset-full
#Advertise=1000baset-half
#Advertise=1000baset-full



 - Jonas

#927397#55

Date:: 2020-08-08 17:38:32 UTC

From:

To:

Quoting Jonas Smedegaard (2019-04-26 09:54:37)

Concretely, wiring affects MDI/MDI-X, commonly emulated transparently in
both hosts and switches nowadays (because it is an optional part of the
standard for gigabit ethernet), with no need for actually using a
"cross-over cable".

I do not suspect any flaws in MDI/MDI-X handling itself, but indirectly
the _need_ for NDI-X matters anyway: The cause for the packet loss
issues is likely a timing issue. Ethernet is tied to a clock signal fed
from one end of the wiring - the "master". When using a switch the
switch end of the wiring becomes master, but in "cross-over" wiring (no
matter if a cross-over cable is used or whichever of the host PHYs
emulate cross-over by flipping from the normal MDI to MDI-X) it is
_undefined_ which end gets becomes master.

It can seem more reliable to setup a minimal test involving only two
hosts and a cable, but in reality that introduces less reliable results
than using a switch in-between.

Sunil: It would be helpful to know who from Olimex you talked to, so
that we can try get back to them and figure out if their tests leading
to 20% failure was done "head-to-head" (where it is undefined which end
becomes master but perhaps just a fancier chipset at the peer end wins a
random race 80% of the time), or they used a switch (where I cannot
think of such obvious explanation for the 20% failure, and fall back on
"20% of the chips are behaving differently than the rest").

What I am hoping for is to that all chips behave the same - that the 20%
can be explained by the wiring in the test setup.


 - Jonas

#927397#62

Date:: 2021-08-26 18:50:53 UTC

From:

To:

Hi all,

the last days, I tried upgrading two A20-OLinuXino-Lime2 boards to bullseye.
After booting the bullseye kernel, 5.10.46-4 (2021-08-03), the network did
not work anymore: The interface shows up (IIRC, even a link is detected),
but any attempts to connect (DHCP) fail.

After playing with different delays following [1], I succeeded with
CONFIG_GMAC_TX_DELAY=4, where both boards seem to work fine. I also played a
bit with iperf3:  At higher bitrates (> approx. 200Mbits/s) more and more
retries seem to be needed.  So the issue this bug is about seems to affect
also the G2 revision boards that worked fine so far.

Best regards,

  Andi


[1] https://wiki.debian.org/InstallingDebianOn/Allwinner#Olimex_A20-OLinuXino-LIME2_rev._K.3B_rev._G2_and_bullseye_kernel

#927397#67

Date:: 2022-06-09 10:25:01 UTC

From:

To:

This bug was fixed since u-boot 2022.04 - specifically in git commit
f11513d9: https://source.denx.de/u-boot/u-boot/-/commit/f11513d

 - Jonas

#927397#72

Date:: 2022-06-09 10:34:25 UTC

From:

To:

Quoting Jonas Smedegaard (2022-06-09 12:25:01)
https://source.denx.de/u-boot/u-boot/-/commit/85da558

 - Jonas

#927397 u-boot: Olimex Lime2 Rev.F-G2: severe packet loss w/ gigabit ethernet #927397

Just Reply to ...

Reply to submitter ...

Send control command (Silently)

Set Architecture Tags (Silently)