Description:
We have identified a forward-compatibility regression in the md
subsystem introduced between Linux Kernel 6.18 and 6.19. When creating a
standard RAID 1 array using mdadm, the resulting superblock generated
under Kernel 6.19 is violently rejected by older enterprise Linux
kernels (e.g., older LTS kernels with version 6.1) with an -EINVAL
argument error during assembly.

A strict A/B test was performed using Debian Live (based on Sid)
environments. The exact same version of mdadm (4.5-5) and the exact same
creation command was used in both environments:
mdadm --create "/dev/md127" --force --run --level=raid1 --raid-devices=2
--metadata=1.2 --data-offset=34816 --bitmap=internal --assume-clean 
/dev/nvme0n1p2 /dev/nvme1n1p2

Test 1 (Success):
* Environment: Kernel 6.18.15+deb14-amd64
* Result: Array creates successfully. When the machine reboots into the
older guest OS, the legacy kernel assembles the array normally.

Test 2 (Failure/Regression):
* Environment: Kernel 6.19.6+deb14-amd64
* Result: Array creates successfully in the live environment. However,
when the machine reboots into the older guest OS, the legacy kernel
rejects the component drives with:
   mdadm: failed to add /dev/nvme1n1p2 to /dev/md/0: Invalid argument.
   mdadm: failed to add /dev/nvme0n1p2 to /dev/md/0: Invalid argument.
   mdadm: /dev/md0 assembled from 0 drives

Conclusion:
Kernel 6.19 appears to be injecting a new feature flag, modifying the
bitmap format, or altering the superblock structure during creation in a
way that older md_mod drivers cannot parse, breaking backward
compatibility for disaster recovery tools restoring older operating systems.

Steven

Yes, it's related to this, most likely:
md0: echo current LBS to md/logical_block_size to prevent data loss issues
from LBS changes.
               Note: After setting, array will not be assembled in old
kernels (<= 6.18)

So, if you create a new one it will have the md/logical_block_size already
set. Is there no warning when creating a new md array in 6.19?

Yes, it's related to this, most likely:
md0: echo current LBS to md/logical_block_size to prevent data loss issues
from LBS changes.
               Note: After setting, array will not be assembled in old
kernels (<= 6.18)

So, if you create a new one it will have the md/logical_block_size already
set. Is there no warning when creating a new md array in 6.19?

Hi,

I believe this is the same as reported upstream in
https://lore.kernel.org/all/b3e941b0-38d1-4809-a386-34659a20415e@gmail.com/

In my understanding it was decided that that new created arrays will
not be possible to assemble with older kernels. Future stable series
kernels might get an additional patch to make it possible that array
with default lbs can be still assembled again in old kernels.

Regards,
Salvatore