- Package:
- fai-setup-storage
- Source:
- fai-setup-storage
- Submitter:
- Date:
- 2024-02-17 04:48:03 UTC
- Severity:
- normal
- Tags:
(setup-storage identifies itself as 6.0.3; to be precise it's the
version pre-installed in the faicd64-ubuntu-only_6.0.3.iso
CD image. The bug was also present in 5.10.1/faicd64-ubuntu-only_5.10.1+3.iso
but we were too harried back then to remember to report the problem;
sorry about that.)
Setup-storage expects a numeric /dev/mdN device to have at least
three digits: /dev/md123 or /dev/md000 is expected, /dev/md0 or
/dev/md12 is not. It's mistaken: /dev/md0 and /dev/md12 (and any
md device with one or two digits) are valid to Linux.
The effect is that if told to partition /dev/md0, setup-storage
doesn't understand that partition 1 on that device is /dev/md0p1;
it tries /dev/md01. That in turn is diagnosed as an invalid
device name; setup-storage aborts; so does the install.
The problem seems to be in two regexps in setup-storage/Init.pm,
which use md\d{3,} (i.e. match three or more digits). I worked
around the problem by changing it to md\d+ (i.e. one or more digits).
Maybe what was originally intended was md\d{1,3} (at least one digit,
at most three), and that would work for now because (as I understand
it) the current md implementation can't handle more than 512 arrays;
but it seems wiser to lift the limit as future-proofing.
Here's a diff -c showing what I did to get our installs working:
====
*** Init.pm.stock Wed Aug 23 13:51:21 2023
--- Init.pm Wed Aug 23 13:52:14 2023
***************
*** 207,213 ****
return (1, "/dev/$1", $2);
}
elsif ($dev =~
! m{^/dev/(loop\d+|cciss/c\d+d\d+|ida/c\d+d\d+|md\d{3,}|md/\w+\d*|rd/c\d+d\d+|ataraid/d\d+|etherd/e\d+\.\d+|nvme\d+n\d+|mmcblk\d+)(p(\d+))?$})
{
defined($2) or return (1, "/dev/$1", -1);
return (1, "/dev/$1", $3);
--- 207,213 ----
return (1, "/dev/$1", $2);
}
elsif ($dev =~
! m{^/dev/(loop\d+|cciss/c\d+d\d+|ida/c\d+d\d+|md\d+|md/\w+\d*|rd/c\d+d\d+|ataraid/d\d+|etherd/e\d+\.\d+|nvme\d+n\d+|mmcblk\d+)(p(\d+))?$})
{
defined($2) or return (1, "/dev/$1", -1);
return (1, "/dev/$1", $3);
***************
*** 289,295 ****
sub make_device_name {
my ($dev, $p) = @_;
$dev .= "p" if ($dev =~
! m{^/dev/(loop\d+|cciss/c\d+d\d+|ida/c\d+d\d+|md\d{3,}|md/\w+\d*|rd/c\d+d\d+|ataraid/d\d+|etherd/e\d+\.\d+|nvme\d+n1|mmcblk\d+)$});
$dev .= $p;
internal_error("Invalid device $dev") unless (&FAI::phys_dev($dev))[0];
return $dev;
--- 289,295 ----
sub make_device_name {
my ($dev, $p) = @_;
$dev .= "p" if ($dev =~
! m{^/dev/(loop\d+|cciss/c\d+d\d+|ida/c\d+d\d+|md\d+|md/\w+\d*|rd/c\d+d\d+|ataraid/d\d+|etherd/e\d+\.\d+|nvme\d+n1|mmcblk\d+)$});
$dev .= $p;
internal_error("Invalid device $dev") unless (&FAI::phys_dev($dev))[0];
return $dev;
====
And here's a snippet from fai.log showing the original error.
We have a hooks/partition.CLASS that initializes the md (after
using some local heuristics to decide which disks may be used
safely), then sets disklist=/dev/md, then returns to task_partition
which (with the original Init.pm) complaints and aborts:
====
Starting setup-storage 3.0
Using config file: /var/lib/fai/config/disk_config/UBUNTU_22_04
INTERNAL ERROR in setup-storage:
Invalid device /dev/md01
Please report this error to the Debian Bug Tracking System.
at /usr/share/fai/setup-storage/Init.pm line 294, <$config_file> line 1.
FAI::make_device_name("/dev/md0", 1) called at /usr/share/fai/setup-storage/Parser.pm line 318
FAI::init_part_config("primary") called at (eval 86) line 7624
Parse::RecDescent::namespace000001::type(Parse::RecDescent=HASH(0x560538a6c6c8), "primary\x{9}/\x{9}72000\x{9}ext4\x{9}rw,errors=remount-ro\x{9}createopts=\"-m3\"\x{a}pr"..., 1, undef, CODE(0x5605393d1048), undef) called at (eval 86) line 9569
Parse::RecDescent::namespace000001::volume(Parse::RecDescent=HASH(0x560538a6c6c8), "primary\x{9}/\x{9}72000\x{9}ext4\x{9}rw,errors=remount-ro\x{9}createopts=\"-m3\"\x{a}pr"..., 1, undef, CODE(0x5605393e24b0), undef) called at (eval 86) line 716
Parse::RecDescent::namespace000001::config(Parse::RecDescent=HASH(0x560538a6c6c8), "primary\x{9}/\x{9}72000\x{9}ext4\x{9}rw,errors=remount-ro\x{9}createopts=\"-m3\"\x{a}pr"..., 1, undef, CODE(0x5605393d0030), undef) called at (eval 86) line 3373
Parse::RecDescent::namespace000001::line(Parse::RecDescent=HASH(0x560538a6c6c8), "primary\x{9}/\x{9}72000\x{9}ext4\x{9}rw,errors=remount-ro\x{9}createopts=\"-m3\"\x{a}pr"..., 1, undef, CODE(0x5605392109a0), undef) called at /usr/share/perl5/Parse/RecDescent.pm line 3275
Parse::RecDescent::_parserepeat(Parse::RecDescent=HASH(0x560538a6c6c8), "# \$Header: /admin/cvsroot/debian/FAI/fai/disk_config/UBUNTU_2"..., CODE(0x560538bc8920), 0, 100000000, undef, Parse::RecDescent::Expectation=HASH(0x5605393d1f70), CODE(0x5605392109a0), ...) called at (eval 86) line 2621
Parse::RecDescent::namespace000001::file(Parse::RecDescent=HASH(0x560538a6c6c8), "# \$Header: /admin/cvsroot/debian/FAI/fai/disk_config/UBUNTU_2"..., undef, undef, CODE(0x5605384dd830), undef) called at /usr/share/perl5/Parse/RecDescent.pm line 3237
Parse::RecDescent::AUTOLOAD(Parse::RecDescent=HASH(0x560538a6c6c8), "# \$Header: /admin/cvsroot/debian/FAI/fai/disk_config/UBUNTU_2"...) called at /usr/share/fai/setup-storage/Parser.pm line 1222
FAI::run_parser(GLOB(0x56053851a7b0)) called at /usr/sbin/setup-storage line 160
This is your disk_config file:
# $Header: /admin/cvsroot/debian/FAI/fai/disk_config/UBUNTU_22_04,v 1.1 2023/08/02 01:14:28 norman Exp $
#
#
# NOTE: this file is in the new format. See setup-storage(8) on deb4.
disk_config disk1 disklabel:gpt-bios bootable:1
primary / 72000 ext4 rw,errors=remount-ro createopts="-m3"
primary /boot/efi 1000 vfat rw,umask=027
primary swap 2000 swap sw,discard
primary /var 15000 ext4 rw,nodev createopts="-m5"
primary /tmp 15000 ext4 rw,nodev,nosuid createopts="-m1"
primary /data 0- ext4 rw,nodev,nosuid createopts="-m1"
Error in task partition. Code: 710
Traceback: task_error task_partition task task_install task task_action task main
====
Glad to dig up further details if it will help.
Thanks,
Norman Wilson
University of Toronto
Hi,
I guess the \d{3,} regex comes from the fact, that during an initial
FAI installation (booting the FAI environment) the md devices are
always higher that 127 IIRC. If you use setup-storage on a normal
Debian system, this is not true any more.
I will think about changing that.
I've check with the upcoming FAI version, and there the md device names do start with md0. I've already added your fix to the git repo.
I've check with the upcoming FAI version, and there the md device names do start with md0. I've already added your fix to the git repo.
We believe that the bug you reported is fixed in the latest version of
fai, which is due to be installed in the Debian FTP archive.
A summary of the changes between this version and the previous one is
attached.
Thank you for reporting the bug, which will now be closed. If you
have further comments please address them to 1050383@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.
Debian distribution maintenance software
pp.
Thomas Lange <lange@debian.org> (supplier of updated fai package)
(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)
Format: 1.8
Date: Sat, 20 Jan 2024 10:39:45 +0100
Source: fai
Architecture: source
Version: 6.2~a1
Distribution: experimental
Urgency: low
Maintainer: Thomas Lange <lange@debian.org>
Changed-By: Thomas Lange <lange@debian.org>
Closes: 1030312 1041609 1050383 1054262 1054263
Changes:
fai (6.2~a1) experimental; urgency=low
.
[ Thomas Lange ]
* get-config-dir-svn: remove stray space, Closes: #1041609
* Init.pm: allow more md device names, Closes: #1050383,
fix regex for nvme devices with namespaces
* get-config-dir-detect: new method
* new script mk-data-partition, add option -c
* fai: drop sysv code for starting udev,
if a partition named MY-DATA exists mount it to /media/data
* task_chboot: do not call if called by fai-diskimage
* fai-diskimage: add option -C, use zstd for qcow2 format
* fai-savelog: add optional directory to -l
* task_instsoft: install packages from /media/data/pkgs
* task_savelog: save logs to local data partition if it exists
* terminal-wrapper: add support for running inside a screen or tmux
session
* fai-cd: redirect errors from dpkg call, preserve hardlinks when
copying, write FAI version into FAI-CD, use custom fs.lst for grub
* fai-scan.sh: redirect dialog output to console, needed for systemd
* grub.cfg: add partition regex and additional path to EFI environment,
fix wrong test, ignore FAI CD, print warning if no grub.cfg is found
* grub.cfg.live: add config for live ISO
* fai-cd.8: add example how to create a live ISO
* fai-kvm: add boot from USB, do not add disks that do not exist
* task_dirinstall: print size of created file system
* task_chboot: handle FAI_LOGPROTO=none
* fai-cd: handle symlink to nfsroot, add unicode font
* mkdebmirror: switch to bookworm
* fai-mirror: add non-free-firmware
* install_packages: use apt instead of aptitude for just unpacking,
remove unsupported aptitude option '--allow-change-held-packages',
Closes: #1030312
* conf/NFSROOT: add uuidgen, more nonfree firmware
* fai-mount-disk: ignore USB stick we booted from, ignore
/dev/mapper/live-base
* fai-make-nfsroot: add support for symlinks, adjust modules for systemd
* setup_storage: add variable FAI_KEEP_CRYPTKEYFILE
* fai-mirror.1: explain when -B should not be used
* fai-disk-info: move multiple functions into subroutines
* package_config/FAISERVER: remove grub-pc, Closes: #1054263, #1054262
remove nscd
* package_config/DEBIAN: add bash-completion
* disk_config: increase size of /boot
* rc.local/FAISERVER: add information messages, always move rc.local to
/var/tmp, restart apt-cacher-ng
* FAISERVER/10-conffiles: more options for apt-cacher-ng
* mk-basefile: add Alma Linux
* add class LIVEISO
* updatebase: use ctam()
* hook/subroutines: define two cleanup_.. functions
* instsoft.DEBIAN: use zstd for dracut
* subroutines: do not use bind for mounting pseudo filesystems in ctam()
add set_bootstick,
add multiple functions for generating disk lists
grepv_disks()
grep_disks()
notmatchdisks()
matchdisks()
smallestdisk()
largestdisk()
all_disks_by_size()
all_disks_and_size()
once_only()
checkdisk()
disks_by_id()
.
[ Henning Glawe ]
* fai-mount-disk: ignore partition type "zfs_member"
* mount2dir: ignore nfs4 and cephfs
.
[ Moritz 'Morty' Strübe ]
* add support for systemd
Checksums-Sha1:
e945d9b33c6e3d26f196666ffd56cd0a33fa993a 2074 fai_6.2~a1.dsc
ebdfee42fb7674a1a1e75158a942a33fb75a9902 299544 fai_6.2~a1.tar.xz
8aef0e12f5b625c392b8dc5d21df995094ff0fe5 16871 fai_6.2~a1_amd64.buildinfo
Checksums-Sha256:
d9bd419d4cd3bd5bf0d18906dcf6248bfbe34d67a36c186b67e070c2970ff862 2074 fai_6.2~a1.dsc
650357218ead4b6a16d7569c17ab900609e06192f6546d9fb7f2d29b7acd958f 299544 fai_6.2~a1.tar.xz
fdfa9bdf684796a1bee49f4b16b03aadf6821e7e1d39d03ffac855eb0b790f13 16871 fai_6.2~a1_amd64.buildinfo
Files:
99cb537ffe3b8a0004be9213af9015fa 2074 admin optional fai_6.2~a1.dsc
12b6199689be6965967b94bffcc50fe7 299544 admin optional fai_6.2~a1.tar.xz
1e6c99b984bc6a10c6d6bffe8d16f2ba 16871 admin optional fai_6.2~a1_amd64.buildinfo
-----BEGIN PGP SIGNATURE-----
iQJFBAEBCAAvFiEEsR7jJz9rLetSjJPaK/jZ/gdLzeQFAmWrlWsRHGxhbmdlQGRl
Ymlhbi5vcmcACgkQK/jZ/gdLzeQ4PQ//WblM41x4zIv7NCywDhTuVDF7WjrJ4SL8
KLJMDWdlTjmuNOKWUb/ZAYPU83Vcr2irYyx29uPaEQ0MFo5K1Sxlajirx2T2fZbf
Pah1KR8pJEODCEyArS24faYm/NSq7jVEJaZu7VYPxSnumadKpdpbWW9joFfVvZZ6
q/ZwS23muqW7iDP4Xi/ohuXFp80K2el+8pOFyMtzIiEmOit1RA+O+ZZCkC6/X+sX
OtdnvXGzU6beb0kOUQryDqgG68RWg+VkNGvdtZPCQ+ac1dJ15Wc26rDHHcr/f9+t
sHO1Gv1firrcyCabkoNQQ9SP1TeRDR21H+TMNAmKBG8vVI39GmSdg9hYhMn6OgBi
KYWY5fUGB1m+FmZitVe4wNG2FNmFE21D7EVJDrANib24I9uK+EyLeE25SiH4PHT8
qg+aRY46xuWwrLXnvkxX9h0bS/UENoLttdJ4LiCVgTRWRfUrtW1Nv/vvx2vUbHGh
uRPhLB3xFIhCEt2bmaeLgwbawrat4VqYKTGb8mZuwpBU/gObO+RUsN7AKQKmPh31
5XvFVlqltOHUAHH1OVgo6gHxo3qMWJwvGefni5SF8RMZqJx4/wDhUuNwCzHTcHsM
6zStTdspIVCQu7R319BK4OtNwUkh7U/tHX9XlTifnIIWp70jzL9XXE8+Of0hScpo
EB1MV0r6ISw=
=A8DC
-----END PGP SIGNATURE-----
We believe that the bug you reported is fixed in the latest version of
fai, which is due to be installed in the Debian FTP archive.
A summary of the changes between this version and the previous one is
attached.
Thank you for reporting the bug, which will now be closed. If you
have further comments please address them to 1050383@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.
Debian distribution maintenance software
pp.
Thomas Lange <lange@debian.org> (supplier of updated fai package)
(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)
Format: 1.8
Date: Sun, 21 Jan 2024 00:23:15 +0100
Source: fai
Architecture: source
Version: 6.2
Distribution: unstable
Urgency: low
Maintainer: Thomas Lange <lange@debian.org>
Changed-By: Thomas Lange <lange@debian.org>
Closes: 1030312 1041609 1050383 1054262 1054263
Changes:
fai (6.2) unstable; urgency=low
.
[ Thomas Lange ]
* get-config-dir-svn: remove stray space, Closes: #1041609
* Init.pm: allow more md device names, Closes: #1050383,
fix regex for nvme devices with namespaces
* get-config-dir-detect: new method
* new script mk-data-partition, add option -c
* fai: drop sysv code for starting udev,
if a partition named MY-DATA exists mount it to /media/data
* task_chboot: do not call if called by fai-diskimage
* fai-diskimage: add option -C, use zstd for qcow2 format
* fai-savelog: add optional directory to -l
* task_instsoft: install packages from /media/data/pkgs
* task_savelog: save logs to local data partition if it exists
* terminal-wrapper: add support for running inside a screen or tmux
session
* fai-cd: redirect errors from dpkg call, preserve hardlinks when
copying, write FAI version into FAI-CD, use custom fs.lst for grub
* fai-scan.sh: redirect dialog output to console, needed for systemd
* grub.cfg: add partition regex and additional path to EFI environment,
fix wrong test, ignore FAI CD, print warning if no grub.cfg is found
* grub.cfg.live: add config for live ISO
* fai-cd.8: add example how to create a live ISO
* fai-kvm: add boot from USB, do not add disks that do not exist
* task_dirinstall: print size of created file system
* task_chboot: handle FAI_LOGPROTO=none
* fai-cd: handle symlink to nfsroot, add unicode font
* mkdebmirror: switch to bookworm
* fai-mirror: add non-free-firmware
* install_packages: use apt instead of aptitude for just unpacking,
remove unsupported aptitude option '--allow-change-held-packages',
Closes: #1030312
* conf/NFSROOT: add uuidgen, more nonfree firmware
* fai-mount-disk: ignore USB stick we booted from, ignore
/dev/mapper/live-base
* fai-make-nfsroot: add support for symlinks, adjust modules for systemd
* setup_storage: add variable FAI_KEEP_CRYPTKEYFILE
* fai-mirror.1: explain when -B should not be used
* fai-disk-info: move multiple functions into subroutines
* package_config/FAISERVER: remove grub-pc, Closes: #1054263, #1054262
remove nscd
* package_config/DEBIAN: add bash-completion
* disk_config: increase size of /boot
* rc.local/FAISERVER: add information messages, always move rc.local to
/var/tmp, restart apt-cacher-ng
* FAISERVER/10-conffiles: more options for apt-cacher-ng
* mk-basefile: add Alma Linux
* add class LIVEISO
* updatebase: use ctam()
* hook/subroutines: define two cleanup_.. functions
* instsoft.DEBIAN: use zstd for dracut
* 50-misc: add test if $release is empty
* subroutines: do not use bind for mounting pseudo filesystems in ctam()
add set_bootstick,
add multiple functions for generating disk lists
grepv_disks()
grep_disks()
notmatchdisks()
matchdisks()
smallestdisk()
largestdisk()
all_disks_by_size()
all_disks_and_size()
once_only()
checkdisk()
disks_by_id()
.
[ Henning Glawe ]
* fai-mount-disk: ignore partition type "zfs_member"
* mount2dir: ignore nfs4 and cephfs
.
[ Moritz 'Morty' Strübe ]
* add support for systemd
Checksums-Sha1:
5479aa681c2c182ec697b2faf56b3010cc141f21 2062 fai_6.2.dsc
e5b2b6d277042f9503ed6f67b64d4f8f1cac5299 299524 fai_6.2.tar.xz
bd0ede637599596a7cada62f9416e3215d3e5c83 16805 fai_6.2_amd64.buildinfo
Checksums-Sha256:
457371e0c2fb66f1085bafc9b09256648e6c746d9c51c3322d0464e11f0cc2d1 2062 fai_6.2.dsc
82f83824e859f989580db0f53b605b98f868985dca30c10fc9a6dd577934ad3e 299524 fai_6.2.tar.xz
d4dc5f80b1be839ca6c7f6c08d41f01e6da0d29065e60d84eb136f4aa01d38c0 16805 fai_6.2_amd64.buildinfo
Files:
52d6e9852555169fb86953ecf3f4ca7b 2062 admin optional fai_6.2.dsc
19db48a8cd30a5330a40a5ac7a9a5fba 299524 admin optional fai_6.2.tar.xz
85f275ac9e63b196d8c049edb7c891ca 16805 admin optional fai_6.2_amd64.buildinfo
-----BEGIN PGP SIGNATURE-----
iQJFBAEBCAAvFiEEsR7jJz9rLetSjJPaK/jZ/gdLzeQFAmWsV4URHGxhbmdlQGRl
Ymlhbi5vcmcACgkQK/jZ/gdLzeQjNg/8D+mCPRfLI+GsR9O/PFqO6IdhsVmu4o1z
Wfrj8Aax1ilPQj7lbj4bWeVcrpuQ9xsCqZ5QZNJx+yOzkniQOhlSxQlnp8jXwR4o
VXq85Ygjv/sifGDCsZsssD1AnLSLxiVod8a11Ca3nuvilp/DaQkNA3XgLoSVsQaZ
dNguIfXAL3DLRpkE7gdzxUmL69XlR/zc/3/R8bWqnUFkXrigWYFDYa6V8O5trUrU
JgFRgbKaWpoUd1v8zp0u1jWTVQOGxVhkVhSBroeOVTnFhxYXwk2grVZ1dZkX1og3
JnrTqNAk+GgkQz1RXpdYrME+CEFPp8lmTKwhEzr7mH8Ahk32iHJFd6SUvb/nHaC9
5TBctFL2IfPA6F6cz3YAuNiH83KFslPIWEwEEXILKVoWqk64WAearrJ8LLSOTrE4
WFdog+Ot0M6fVku03TLe2WllTa10HlpW7aMA53KShYNY54OmaQ6OZ1qHvRtjmz/b
W2RuOXMck+bdzngFllMfVrZ712ECScVpZwOwyGiXEErnXhJ4J0nw2nQMmyDB8PM3
J9Qcgy6uCeMS8XIwBkBImcE22yk+SJUHuK8sLtHNU1oMgtzi4HbTq1fuMSE4ieXN
Cf+Yg7RSs9RO4iIPR39ybGz9pvaUkUj5XOUESE/fWUD8dK7HGnh7VVhz+PiXps20
+SHUhyN3emY=
=/kni
-----END PGP SIGNATURE-----
Hi, I will revert this fix in FAI 6.2.1, because it broke other things. Our LVM on soft RAID example does not work any more. You application of using a hook for creating a md device and then set disklist=/dev/md is not what setup-storage supports. Why not use your heuristic and create then the disklist (of plain disks) and let setup-storage create the md devices? I'm not sure if we can fix your problem without breaking the normal configs.