Dear Maintainer,


I created a virtual server with 2 disks of the same size. (Problem wil
be the same on a physical server with 2 disks of the same size)
I did a minimum installation with :

- a partition on the whole disk on each disk of type LVM. (/dev/vda1,
/dev/vda2)
- a volume group with name rootvg with /dev/vda1
- Everything on lvm,  with /dev/rootvg/slashlv mounted on / and
/dev/rootvg/swaplv for swap (a minimum partitioned system, I create more
logical volume on my servers)
- grub on /dev/vda
And did a reboot after the installation...
I can work on the server without problem...

After a successul login under root, I did :
- vgextend /dev/rootvg /dev/vdb1
- lvconvert --type raid1 -m 1 /dev/rootvg/slashlv
- lvconvert --type raid1 -m 1 /dev/rootvg/swaplv
- wait for the raid to be completely rebuilded (I checked with the "lvs"
command)

I can work on the server without problem...

But If I reboot, grub can't regnosize my installation...  The message
(recopied manually) is :
"Booting from Hard Disk...
error: disk
`lvmid/Gwx3dG-v00h-9hJU-FVuk-Il8Z-bFUK-i6DkfU/KpVdER-OEMM-MWik-vKwM-hWxp-MeJf-LMnsf7'
not found.
Entering rescue mode...
grub rescue>"
So I try to reboot with the rescue mode but the CD of debian lack the
module "dm-raid" on the kernel part. (Probably on a udeb package but
don't know yet exactly which one, so I created this bug report against
debian-installer), so I cannot access my disk anymore !!! Imagine you do
this on a production server and you did'nt reboot before a long time.

For the record, I already filled a bug report for the grub problem
 on http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=782591

Whyever would you use lvm raid rather than md raid with lvm on top?

As far as I understand it, lvm raid is very inflexible and makes it hard
to ever make changes to the lvm in the future, while if you use md raid
you are using something that is very flexible as raid and can easily be
extended, and lvm on top of that is very flexible too.

Basically your chosen setup is very unusual, and as far as I understand
it highly discouraged.  It will also be very hard to find anyone to help
you if there are problems because it is so unusual (the normal setup
works better and makes more sense, so that is what almost everyone uses).

Whyever would you use lvm raid rather than md raid with lvm on top?

As far as I understand it, lvm raid is very inflexible and makes it hard
to ever make changes to the lvm in the future, while if you use md raid
you are using something that is very flexible as raid and can easily be
extended, and lvm on top of that is very flexible too.

Basically your chosen setup is very unusual, and as far as I understand
it highly discouraged.  It will also be very hard to find anyone to help
you if there are problems because it is so unusual (the normal setup
works better and makes more sense, so that is what almost everyone uses).

Hi,

Le 22/04/2015 03:47, Lennart Sorensen a écrit :

Because I prefer do it this way... lvm raid is more the traditionnal
unix way of doing it (AIX and HP-UX for sure)
Why it is very inflexible ? It found it quite flexible.

Perhaps the md raid support is there from a longer time because raid1
support (which is different from mirror support) in the kernel (and in
lvm2 tools) came later... But things changes and I saw a lot of good
things coming in LVM like thin lvm support or cache lvm support. Read
redhat docs, such things are supported by RedHat since RedHat Enterprise
6.4 or such.

Regards,

Hi,

Le 22/04/2015 03:47, Lennart Sorensen a écrit :

Because I prefer do it this way... lvm raid is more the traditionnal
unix way of doing it (AIX and HP-UX for sure)
Why it is very inflexible ? It found it quite flexible.

Perhaps the md raid support is there from a longer time because raid1
support (which is different from mirror support) in the kernel (and in
lvm2 tools) came later... But things changes and I saw a lot of good
things coming in LVM like thin lvm support or cache lvm support. Read
redhat docs, such things are supported by RedHat since RedHat Enterprise
6.4 or such.

Regards,

That doesn't mean it is a good idea.  Those two certainly qualify as
the most unpleasant unix systems I have ever had to work with.

You can't easily add disks to the volume group in raid mode (as far as
I recall the documentation last I looked at it).  There are certainly
restrictions on how to you expand volumes when lvm raid is in use.

Well certainly grub understands md raid, and it understands some lvm.
It might not understand lvm with raid.  Following the grub development
mailing list for a number of years now, I can say I have never seen
anyone ask about it or even mention it.  I think it is also the first
time I have seen it mentioned on any of the debian lists I follow.

Certainly using lvm on top of md raid has worked for years for me and
the debian rescue knows how to deal with is, as does knoppix and lots
of other rescue systems.  The lvm raid, no idea.  When I looked at it
a few years ago it cretainly sounded like a bad option so I avoided it.

That doesn't mean it is a good idea.  Those two certainly qualify as
the most unpleasant unix systems I have ever had to work with.

You can't easily add disks to the volume group in raid mode (as far as
I recall the documentation last I looked at it).  There are certainly
restrictions on how to you expand volumes when lvm raid is in use.

Well certainly grub understands md raid, and it understands some lvm.
It might not understand lvm with raid.  Following the grub development
mailing list for a number of years now, I can say I have never seen
anyone ask about it or even mention it.  I think it is also the first
time I have seen it mentioned on any of the debian lists I follow.

Certainly using lvm on top of md raid has worked for years for me and
the debian rescue knows how to deal with is, as does knoppix and lots
of other rescue systems.  The lvm raid, no idea.  When I looked at it
a few years ago it cretainly sounded like a bad option so I avoided it.

Le 2015-04-22 14:20, Lennart Sorensen a écrit :

Taste is personnal... There are advantages and inconvenients in all of
them (Linux/AIX/HP-UX/True64/Solaris/FreeBSD/whatever)

FOR AIX vs linux some examples :

some pro :
smitty is nicely implemented (there was linuxconf in the past days of
linux but not maintained anymore)
mksysb is something nice to use to restore a system or to install a
system with the same configuration...
alternate install is something interesting when upgrading from an OS
version to another one (using an alternate lvm mirror)

some con :
odm is something you should not touch without IBM advise, etc..
dependencies in packages are too hard and upgrading a package tend to
have everything upgrading...

But we are not here to speak about proprietary... but about Linux, so I
will not elaborate more on that.

I just add a disk on my test partition... Done a pvcreate on it, done a
vgextend on it without problem.
Done a lvextend on a logical partition... Resize my slashlv file system.
No problem at all. Did you remake some recent tests ? What is the
limitation you are speaking about ? The one I know is that you can't use
lvm raid1 in a cluster yet. (It's written in the RedHat Enterprise 7
Logical Volume Documentation)

You should have read my bug report about grub... There is at least one
another person than me that create the bug report ;-)
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=782591
http://savannah.gnu.org/bugs/?func=detailitem&item_id=44534
http://lists.gnu.org/archive/html/bug-grub/2015-04/msg00003.html

and you should probably read more carefully the list :
http://lists.gnu.org/archive/html/grub-devel/2014-08/msg00024.html

Some years ago you probably test the mirror type... which I agree is not
a so good solution.
I speak about the raid1 type which is different

raid1 is there since 2013 (there's no clear appareance of the first
commit related) :
https://git.fedorahosted.org/cgit/lvm2.git/tree/WHATS_NEW

For mirror, the mirroring is rebuilt at each reboot
For raid1, the mirroring isn't rebuilt at each reboot

I will see if I can provide a patch... because I think that discussion
is rubbish and code is better. I think it's just an option to add
somewhere to enable the built of this kernel module in an udeb.

For the debian-installer part (for this bug report here I speak about
rescuing) I will make another wishlist bug report (there will be more
impact because there will be some more device with rmeta in their name)

Please take the time to enable this module kernel in the udeb.


Regards,

Le 2015-04-22 14:20, Lennart Sorensen a écrit :

Taste is personnal... There are advantages and inconvenients in all of
them (Linux/AIX/HP-UX/True64/Solaris/FreeBSD/whatever)

FOR AIX vs linux some examples :

some pro :
smitty is nicely implemented (there was linuxconf in the past days of
linux but not maintained anymore)
mksysb is something nice to use to restore a system or to install a
system with the same configuration...
alternate install is something interesting when upgrading from an OS
version to another one (using an alternate lvm mirror)

some con :
odm is something you should not touch without IBM advise, etc..
dependencies in packages are too hard and upgrading a package tend to
have everything upgrading...

But we are not here to speak about proprietary... but about Linux, so I
will not elaborate more on that.

I just add a disk on my test partition... Done a pvcreate on it, done a
vgextend on it without problem.
Done a lvextend on a logical partition... Resize my slashlv file system.
No problem at all. Did you remake some recent tests ? What is the
limitation you are speaking about ? The one I know is that you can't use
lvm raid1 in a cluster yet. (It's written in the RedHat Enterprise 7
Logical Volume Documentation)

You should have read my bug report about grub... There is at least one
another person than me that create the bug report ;-)
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=782591
http://savannah.gnu.org/bugs/?func=detailitem&item_id=44534
http://lists.gnu.org/archive/html/bug-grub/2015-04/msg00003.html

and you should probably read more carefully the list :
http://lists.gnu.org/archive/html/grub-devel/2014-08/msg00024.html

Some years ago you probably test the mirror type... which I agree is not
a so good solution.
I speak about the raid1 type which is different

raid1 is there since 2013 (there's no clear appareance of the first
commit related) :
https://git.fedorahosted.org/cgit/lvm2.git/tree/WHATS_NEW

For mirror, the mirroring is rebuilt at each reboot
For raid1, the mirroring isn't rebuilt at each reboot

I will see if I can provide a patch... because I think that discussion
is rubbish and code is better. I think it's just an option to add
somewhere to enable the built of this kernel module in an udeb.

For the debian-installer part (for this bug report here I speak about
rescuing) I will make another wishlist bug report (there will be more
impact because there will be some more device with rmeta in their name)

Please take the time to enable this module kernel in the udeb.


Regards,

Hi,

As a workaround, you can :

- Boot on the Debian CD
- Select Rescue
- Change to a terminal
- cd /tmp
- ar x
/cdrom/pool/main/l/linux/linux-image-3.16.0-4-amd64_3.16.7-ckt7-1_amd64.deb
- xzcat data.tar.xz
- cd /tmp/lib/modules/3.16.0-4-amd64/kernel/drivers/md/
- insmod raid10.ko
- insmod dm-raid.ko
- change back to first console and select your root lv

So sure it is only the udeb lacking this kernel module.
I will work to produce a little patch to the good udeb kernel package.

Regards,

jean-yves@lenhof.eu.org <jean-yves@lenhof.eu.org> (2015-05-05):

They are produced by src:linux, you should only have to update a list,
possibly debian/installer/modules/md-modules?

Mraw,
KiBi.

    *


FWIW: I think that at '*' has to be something like

    tar xf data.tar



Groeten
Geert Stappers

Le 2015-05-05 18:56, Geert Stappers a écrit :
Hum...Hum... True that I forget something... my bad.
This should be more something like this :
xzcat data.tar.xz|tar xvf -

Regards,

JYL

Le 2015-05-05 16:52, Cyril Brulebois a écrit :


Yep, I think so... I'm looking at it now.
The question is how to rebuilt a debian CD after.
Will a replacement of kernel package and udeb kernel package in the
pool/l/linux will be sufficient ?
If not is there a description of how to rebuilt the debian CD anywhere ?

Moreover, I think that dm-thin-pool.ko, dm-persistent-data.ko, and
dm-bio-prison.ko should also be added
to have thin provisionning support in rescue mode/during installation.


Regards,

Yes.
Minor advice make it

 xzcat data.tar.xz | tar xf -

So without the verbose option.


Groeten
Geert Stappers

Geert Stappers <stappers@stappers.nl> (2015-05-06):
few .ko's from the kernel package made it work like Jean-Yves wanted it
to, so we can now focus on figuring out what list of .ko's would make
sense to ship in the proper udeb.

Mraw,
KiBi.

jean-yves@lenhof.eu.org <jean-yves@lenhof.eu.org> (2015-05-06):

You're going to end up with GPG check issues if you do so.

I'd suggest, assuming you're working on jessie:

 1. debcheckout debian-installer
 2. install build-depends
 3. drop updated kernel udebs under build/localudebs
 4. build a monolithic image with:
      make -C build build_monolithic USE_UDEBS_FROM=jessie
 5. check in build/dest/MANIFEST.udebs that the said udebs were
    successfully used during the build
 6. use the generated build/dest/monolithic/mini.iso

(TBH I'm not familiar with all those things.)

Mraw,
KiBi.