Dear maintainer,

It seems that debian-cd doesn't consider packages from the 'unreleased'
distribution (on debian-ports architectures) when running debootstrap.
As a result, packages of priority 'required' or 'important' may be
forgotten in the list of the base packages needed by debootstrap.

The patch below runs debootstrap with option '--extra-suite=unreleased'.

As advised in [1], it is made optional with a new environment variable
'DEBOOTSTRAP_USES_UNRELEASED'.

Regards,
JH Chatenet

[1]: https://lists.debian.org/debian-boot/2020/08/msg00405.html

diff -Naur a/debian-cd/CONF.sh b/debian-cd/CONF.sh
--- a/debian-cd/CONF.sh	2020-04-08 01:35:43.000000000 +0200
+++ b/debian-cd/CONF.sh	2020-08-20 22:50:18.416824575 +0200
@@ -21,6 +21,7 @@
 unset SECURITY          || true
 unset PROPOSED_UPDATES  || true
 unset UNRELEASED        || true
+unset DEBOOTSTRAP_USES_UNRELEASED        || true
 unset BOOTDIR           || true
 unset SYMLINK           || true
 unset COPYLINK          || true
@@ -170,6 +171,10 @@
 # Include packages from Debian Ports unreleased
 #export UNRELEASED=1

+# Include packages from Debian Ports unreleased when listing the packages
+# needed by debootstrap. This requires setting UNRELEASED=1
+#export DEBOOTSTRAP_USES_UNRELEASED=1
+
 # Sparc only : bootdir (location of cd.b and second.b)
 # export BOOTDIR=/boot

diff -Naur a/debian-cd/Makefile b/debian-cd/Makefile
--- a/debian-cd/Makefile	2020-04-08 01:35:43.000000000 +0200
+++ b/debian-cd/Makefile	2020-08-20 22:38:59.129456171 +0200
@@ -412,9 +412,12 @@
 				[ -z "$$BINCLUDE" ] || BINCLUDE="--include=$$BINCLUDE"; \
 				BEXCLUDE=`[ -n "$(BASE_EXCLUDE)" ] && cat $(BASE_EXCLUDE) | tr "\n" "," | sed 's!,$$!!g'`; \
 				[ -z "$$BEXCLUDE" ] || BEXCLUDE="--exclude=$$BEXCLUDE"; \
+				[ _$(UNRELEASED) != _1 ] || \
+				[ _$(DEBOOTSTRAP_USES_UNRELEASED) != _1 ] || EXTRA_SUITES="--extra-suites=unreleased"; \
 				debootstrap $(DEBOOTSTRAP_OPTS) --arch $$ARCH \
 				            --print-debs \
 				            $$BINCLUDE $$BEXCLUDE \
+				            $$EXTRA_SUITES \
 				            $(CODENAME) \
 				            $(TDIR)/debootstrap.tmp \
 				            file:$(MIRROR) \
diff -Naur a/debian-cd/tools/make_disc_trees.pl b/debian-cd/tools/make_disc_trees.pl
--- a/debian-cd/tools/make_disc_trees.pl	2019-12-12 12:22:25.000000000 +0100
+++ b/debian-cd/tools/make_disc_trees.pl	2020-08-20 23:56:24.496491302 +0200
@@ -15,7 +15,8 @@

 my %pkginfo;
 my ($basedir, $mirror, $tdir, $codename, $archlist, $mkisofs, $maxcds,
-    $maxisos, $maxjigdos, $extranonfree, $nonfree, $contrib, $use_local);
+    $maxisos, $maxjigdos, $extranonfree, $nonfree, $contrib, $use_local,
+    $use_unreleased, $debootstrap_uses_unreleased);
 my $mkisofs_base_opts = "";
 my $mkisofs_opts = "";
 my $mkisofs_dirs = "";
@@ -86,6 +87,9 @@
 $nonfree = read_env('NONFREE', 0);
 $contrib = read_env('CONTRIB', 0);
 $use_local = read_env('LOCAL', 0);
+
+$use_unreleased = read_env('UNRELEASED',0);
+$debootstrap_uses_unreleased = read_env('DEBOOTSTRAP_USES_UNRELEASED',0);

 my $list = "$tdir/list";
 my $bdir = "$tdir/$codename";
@@ -530,6 +534,13 @@
 	my $p;
 	my $db_error = 0;
 	my $error_string = "";
+	my $extra_suites = "";
+
+	if ($use_unreleased && $debootstrap_uses_unreleased) {
+		print "    debootstrap invoked with option '--extra-suites=unreleased'\n";
+		print LOG "Checking base is installable for $arch : debootstrap invoked with option '--extra-suites=unreleased'\n";
+		$extra_suites = "--extra-suites=unreleased";
+	}

 	open (PLIST, $packages_file)
 		|| die "Can't open Packages file $packages_file : $!\n";
@@ -558,7 +569,7 @@
 		close ELIST;
 	}

-	open (DLIST, "debootstrap --arch $arch --print-debs $codename $tdir/debootstrap_tmp file:$mirror $debootstrap_script 2>/dev/null | tr ' ' '\n' |")
+	open (DLIST, "debootstrap --arch $arch $extra_suites --print-debs $codename $tdir/debootstrap_tmp file:$mirror $debootstrap_script 2>/dev/null | tr ' ' '\n' |")
 		 || die "Can't fork debootstrap : $!\n";
 	while (defined($p = <DLIST>)) {
         if ($p =~ m/^E:/) {

I have to admit I'm starting to become a bit to wonder why you are trying to
push the use of the "unreleased" distribution.

It's by far not a pressing problem since it affects only a tiny fraction of
packages and those are only cases of workarounds anyway. "unreleased" is
a hack and porters should try to avoid it and get their fixes upstreamed.
I don't think we should further encourage the use of workarounds.

A much more pressing problem is getting support for "contrib" and "non-free"
in Debian Ports so that users have access to firmware during installation.

Furthermore, we really need to switch Debian Ports from Mini-DAK to a proper
DAK instance so that don't run into problems with the lack of cruft in
Debian Ports anymore [1].

Furthermore pressing tasks are:

- update the hfsprogs package and create a udeb for d-i
- update grub-installer and partman-auto to use the hfsprogs udeb on PowerMacs
- create a udeb of the libunwind package required for ia64
- re-introduce old xorg drivers needed for PowerMacs (mach64, r128 etc)
- finish the OpenJDK Zero port to hppa (we are still missing Java on hppa)
- fix GHC on sh4
- add support for HFS (not HFS+) to libburnia so we can get rid of cdrkit
- ...

Adrian

On Fri, 2020-08-21 at 13:31 +0200, John Paul Adrian Glaubitz wrote:
[...]
[...]
[...]

It might be a better choice in the long term to write kernel (KMS)
drivers for these instead.  Those would then be usable by both X and
Wayland servers and should be able to handle suspend and resume
gracefully.

Ben.

Hello,

For information, I submitted a couple of fixes for the unreleased
distribution in debian-cd, it now behaves fine when setting UNRELEASED=1
in CONF.sh: it includes packages from the unreleased distribution in the
sid distribution of the CD. Also, I switched to using LOCAL=1,
LOCALDEBS and UPDATE_LOCAL=1, which is indeed enough to select any few
packages from unreleased for the bootstrap part.

Samuel