Package: debian-installer


Hi,

The netboot tarball misses virtio block drivers.
This makes it unnessecary tiresome
to install Debian VMs on not-Internet-connected server.

More information in next email.
This email is creating the bugreport.


Groeten
Geert Stappers

Why this bugreport was created.

----- Forwarded message from Geert Stappers -----

Date: Thu, 19 Sep 2019 19:15:09 +0200
From: Geert Stappers
To: debian-boot@lists.debian.org
Subject: Re: Can't install a Debian 10 virtual system through netboot
User-Agent: NeoMutt/20170113 (1.7.2)

Thank you for reporting this.  Yes, that is a sincere "thank you".
Because things unknown being broken won't get fixed.

A half year ago (Hi you in Bcc  ;-)
I was told about missing virtio_blk in Debian Stretch.
And was able to reproduce it (no virtio_blk module)

A test with a Debian Buster yield the module present and went on.

So now with the new report I'll have another look at it.


Groeten
Geert Stappers

----- End forwarded message -----

Hi,


Doing:

mkdir newdir
cd newdir
wget http://ftp.nl.debian.org/debian/dists/buster/main/installer-amd64/current/images/netboot/netboot.tar.gz
tar xf netboot.tar.gz
mkdir newerdir
cd newerdir
zcat ../debian-installer/amd64/initrd.gz | cpio -id
find . -name virtio_\*

Gives:
./lib/modules/4.19.0-6-amd64/kernel/drivers/char/virtio_console.ko
./lib/modules/4.19.0-6-amd64/kernel/drivers/virtio/virtio_input.ko
./lib/modules/4.19.0-6-amd64/kernel/drivers/virtio/virtio_pci.ko
./lib/modules/4.19.0-6-amd64/kernel/drivers/virtio/virtio_balloon.ko
./lib/modules/4.19.0-6-amd64/kernel/drivers/virtio/virtio_mmio.ko
./lib/modules/4.19.0-6-amd64/kernel/drivers/virtio/virtio_ring.ko
./lib/modules/4.19.0-6-amd64/kernel/drivers/net/virtio_net.ko



On a totally other system:
$ find /lib/modules/$(uname -r) -name virtio_\*
/lib/modules/4.4.0-121-generic/kernel/drivers/virtio/virtio_input.ko
/lib/modules/4.4.0-121-generic/kernel/drivers/scsi/virtio_scsi.ko


What is needed to get virtio Kernel Objects in the installer initrd.gz


Groeten
Geert Stappers

This is intentional.  netboot is for booting from _network_.  On the
network you got a distribution mirror, otherwise you have problems
installing Debian anyway.

Bastian

Le 20/09/2019 à 07:21, Bastian Blank a écrit :
     Hello Bastian,

In fact the VM is actually booting from network, and has access to a
distribution mirror (copy of DVD-1 of Debian Buster 10.1), but the
debian-installer can't detect VM disk to be partitionned… I don't know
very well debian-installer internals, should it downloads a .deb or
.udeb for disk modules to be able to detect them ? I saw in previous
Debian versions some « virtio-modules » like :
virtio-modules-4.9.0-11-amd64-di, containing virtio_blk module, but it
doesn't exist in Buster, how the installer is supposed to get it ?

      With regards,

         Fred.

It's in scsi-modules-<version>-di now.  (I know it's not a SCSI driver,
but that's where we currently put miscellaneous storage drivers.)

The netboot installer should install that package and then trigger
loading of the module.

Have you updated the netboot installer since the point release?

Ben.

Thing I would like to known is if debian-installer is mirrored.

It is http://ftp.nl.debian.org/debian/dists/buster/main/debian-installer/
usual stuff is in http://ftp.nl.debian.org/debian/dists/buster/main/
Note that both directories have binary-ARCH directories.

Logfiles of the local HTTP mirror can tell.


Groeten
Geert Stappers


P.S.
Please don't send the HTTP server logfiles, analyze them yourself.

     Hello,

Looking at first Buster installation DVD's contents
(debian-10.1.0-amd64-DVD-1.iso found in
https://cdimage.debian.org/debian-cd/current/amd64/iso-dvd/), I can't
find this scsi-modules-<version>-di  package :

$ find /var/www/html/buster_debian_installer/ -name
\*-modules-4.19.0-6\* | xargs -n1 basename | sort
btrfs-modules-4.19.0-6-amd64-di_4.19.67-2_amd64.udeb
compress-modules-4.19.0-6-amd64-di_4.19.67-2_amd64.udeb
crc-modules-4.19.0-6-amd64-di_4.19.67-2_amd64.udeb
crypto-dm-modules-4.19.0-6-amd64-di_4.19.67-2_amd64.udeb
crypto-modules-4.19.0-6-amd64-di_4.19.67-2_amd64.udeb
efi-modules-4.19.0-6-amd64-di_4.19.67-2_amd64.udeb
event-modules-4.19.0-6-amd64-di_4.19.67-2_amd64.udeb
ext4-modules-4.19.0-6-amd64-di_4.19.67-2_amd64.udeb
firewire-core-modules-4.19.0-6-amd64-di_4.19.67-2_amd64.udeb
fuse-modules-4.19.0-6-amd64-di_4.19.67-2_amd64.udeb
i2c-modules-4.19.0-6-amd64-di_4.19.67-2_amd64.udeb
jfs-modules-4.19.0-6-amd64-di_4.19.67-2_amd64.udeb
loop-modules-4.19.0-6-amd64-di_4.19.67-2_amd64.udeb
md-modules-4.19.0-6-amd64-di_4.19.67-2_amd64.udeb
mmc-core-modules-4.19.0-6-amd64-di_4.19.67-2_amd64.udeb
mtd-core-modules-4.19.0-6-amd64-di_4.19.67-2_amd64.udeb
multipath-modules-4.19.0-6-amd64-di_4.19.67-2_amd64.udeb
nbd-modules-4.19.0-6-amd64-di_4.19.67-2_amd64.udeb
nic-modules-4.19.0-6-amd64-di_4.19.67-2_amd64.udeb
nic-pcmcia-modules-4.19.0-6-amd64-di_4.19.67-2_amd64.udeb
nic-shared-modules-4.19.0-6-amd64-di_4.19.67-2_amd64.udeb
nic-usb-modules-4.19.0-6-amd64-di_4.19.67-2_amd64.udeb
nic-wireless-modules-4.19.0-6-amd64-di_4.19.67-2_amd64.udeb
pata-modules-4.19.0-6-amd64-di_4.19.67-2_amd64.udeb
pcmcia-storage-modules-4.19.0-6-amd64-di_4.19.67-2_amd64.udeb
ppp-modules-4.19.0-6-amd64-di_4.19.67-2_amd64.udeb
scsi-nic-modules-4.19.0-6-amd64-di_4.19.67-2_amd64.udeb
sound-modules-4.19.0-6-amd64-di_4.19.67-2_amd64.udeb
squashfs-modules-4.19.0-6-amd64-di_4.19.67-2_amd64.udeb
udf-modules-4.19.0-6-amd64-di_4.19.67-2_amd64.udeb
uinput-modules-4.19.0-6-amd64-di_4.19.67-2_amd64.udeb
xfs-modules-4.19.0-6-amd64-di_4.19.67-2_amd64.udeb


And looking at CD image in
https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/, I find only
the 'netinst' image…

     Fred.


Le 20/09/2019 à 16:19, Ben Hutchings a écrit :

Hi Fred,

Ah, you won't - the contents of it are included in the initramfs
directly, so that the early installer startup code can use those
modules to find the CD/DVD/USB to get the next set of modules that
might be needed.

...

There's also a single CD image for installing a simple desktop
(xfce). We have not made full CD sets for a couple of releases now, as
they're just not very useful due to the limited size of the media.

What exactly are you trying to do, OOI?

[ re-adding the CC :-) ]

Right.

So, let me explain the setup we have. There are different instances of
initramfs created for different needs, and they contain different
things.

1. netboot initramfs

   This is designed to be booted from the network. It includes the
   core of the installer. It includes a small-ish subset of kernel
   modules, basically just what's needed to configure network devices
   and then go and download all the other bits (filesystems, block
   device drivers, more installer modules, etc.) from a Debian mirror.

   If you're using a netboot setup, you will need to include a full
   set of udebs from a Debian mirror in a place where the installer
   can find them. An extra constraint for the network installer here
   is that the kernel module udebs you provide must match *exactly*
   the version of the kernel that you have booted. This is a thing
   that often bites people, hence I mention it.

   The initramfs here directly includes the contents of *some* of the
   udebs, so it will not need to fetch them separately.

2. CD/DVD/USB initramfs

   This is designed to be booted from a disk-like device. It includes
   the core of the installer. It includes a small-ish subset of kernel
   modules, basically just what's needed to locate, configure and
   mount block devices so it can go and find the rest of the needed
   components (network drivers, more installer modules, etc.) from
   those block devices.

   If you're using a Debian-provided CD/DVD/USB based installer, then
   you will already have all the udebs that are needed to go with it,
   as part of that CD/DVD/USB image. The version constraint for the
   module udebs is still there, but you (normally) don't need to worry
   about this as you have a complete set of things in one place.

   Again, the initramfs here directly includes the contents of some of
   the udebs so we do *not* include those udebs in the CD/DVD/USB
   image, to save space - we don't need two copies, after all. This
   includes the "scsi" storage drivers and a few others.

So, this is why you haven't found the virtio modules on the contents
of the DVD image you're using. You'll probably need to grab the rest
of the udebs from a network mirror and share those separately. Look
for

  pool/main/l/linux-signed-amd64/scsi-modules-4.19.0-6-amd64-di_4.19.67-2_amd64.udeb

for current buster for amd64, for example.

I hope that helps to explain?

     Thanks Steve for this clear explanation, I understand now better
how installer is working :-)   !

For my need, I can't get a full Debian mirror for disk space
consideration and lack of Internet access, I must stay with my DVD-1
mirror, so I've look how to modify netboot's initrd.gz to add virtio_blk
(and possible dependencies) inside it : I've updated the initrd.gz and
now, the Debian installer find the virtual disk and starts partitionning
:-) !

I have now 2 other problems :

1/ the first one is that my VM has "only" 512Mo of RAM, and I get a "low
memory" info requiring a manual validation to continue : I've checked in
example-preseed.txt for Buster about an item to skip this warning, with
no luck. I've also searched on the web, finding some posts talking about
"lowmem/info" tag, I've tried for example to add :

d-i lowmem/low note

in my preseed file, without any change, the installer always stops with
the low-memory info…


2/ the second is much serious : the Debian installer fails to install a
kernel package. Looking at log, it's a problem with the DVD-1 mirror not
being [GPG-] signed, so all their packages are rejected :

Sep 26 07:35:26 apt-install: Queueing package keyboard-configuration for
later installation
Sep 26 07:35:26 apt-install: Queueing package console-setup for later
installation
Sep 26 07:35:26 base-installer: Ign:1
http://192.168.254.254/buster_debian_installer buster InRelease
Sep 26 07:35:26 base-installer: Get:2
http://192.168.254.254/buster_debian_installer buster Release [33.5 kB]
Sep 26 07:35:26 base-installer: Ign:3
http://192.168.254.254/buster_debian_installer buster Release.gpg
Sep 26 07:35:26 base-installer: Reading package lists...
Sep 26 07:35:26 base-installer:
Sep 26 07:35:26 base-installer: E: The repository
'http://192.168.254.254/buster_debian_installer buster Release' is not
signed.
Sep 26 07:35:26 base-installer: warning: apt update failed: 100
Sep 26 07:35:26 base-installer: dpkg-divert: warning: diverting file
'/sbin/start-stop-daemon' from an Essential package with rename is
dangerous, use --no-rename
Sep 26 07:35:27 in-target: Reading package lists...
Sep 26 07:35:27 in-target:
Sep 26 07:35:27 in-target: Building dependency tree...
Sep 26 07:35:27 in-target: Package locales is not available, but is
referred to by another package.
Sep 26 07:35:27 in-target: This may mean that the package is missing,
has been obsoleted, or
Sep 26 07:35:27 in-target: is only available from another source
Sep 26 07:35:27 in-target:
Sep 26 07:35:27 in-target: E: Package 'locales' has no installation
candidate
Sep 26 07:35:27 localechooser: error: the command 'validlocale' is not
available
Sep 26 07:35:27 base-installer: info: Found kernels ''
Sep 26 07:35:27 base-installer: error: exiting on error
base-installer/kernel/no-kernels-found

Yet, I have in my preseed.txt the following statement :

# By default the installer requires that repositories be authenticated
# using a known gpg key. This setting can be used to disable that
# authentication. Warning: Insecure, not recommended.
#d-i debian-installer/allow_unauthenticated boolean true
d-i debian-installer/allow_unauthenticated boolean true

How can I tell Debian-installer to use my [unsigned] repository ?


     With regards,

         Fred.

Le 23/09/2019 à 20:38, Steve McIntyre a écrit :