Dear maintainers,
sbuild starting with 0.87.1 fails when build dependencies are missing
from the host system.
E.g. building kde-spectacle:
dpkg-source: info: using options from spectacle/debian/source/local-options: --abort-on-upstream-changes
dpkg-buildpackage: info: source package kde-spectacle
dpkg-buildpackage: info: source version 24.08.0-2
dpkg-buildpackage: info: source distribution unstable
dpkg-buildpackage: info: source changed by Aurélien COUDERC <coucouf@debian.org>
dpkg-checkbuilddeps: error: Unmet build dependencies: libkcolorpicker-qt6-dev libkimageannotator-qt6-dev libopencv-core-dev libopencv-dev libopencv-imgproc-dev libzxing-dev qt6-declarative-private-dev (>= 6.6.0~)
dpkg-buildpackage: warning: build dependencies/conflicts unsatisfied; aborting
dpkg-buildpackage: warning: (Use -d flag to override.)
E: Failed to clean source directory /home/coucouf/hack/debian/kde/gear/spectacle (/home/coucouf/hack/debian/kde/gear/kde-spectacle_24.08.0-2.dsc)
Reverting to 0.87.0 fixes the issue.
I’m running sbuild from inside a git clone of the repo [1].
I have :
$chroot_mode = "schroot";
$schroot = "schroot";
in my ~/.config/sbuild/config.pl.
Maybe I’m missing something so I’ve set the severity as important but
AFAIC this should be RC.
[1] https://salsa.debian.org/qt-kde-team/kde/spectacle
Thanks !
--
Aurélien
Hi,
Quoting Aurélien COUDERC (2024-11-26 08:44:29)
this is the result of a bug fix. In the past, sbuild ran this to clean the
source package before running "dpkg-source -b .":
fakeroot debian/rules clean
This is problematic because
a) fakeroot should not be needed for packages with Rules-Requires-Root
b) fakeroot is buggy and should be avoided
c) we want to get rid of fakeroot in the long run
d) running the clean target requires some packages being installed to run
the tools that do the cleaning, and there was no check if that is the
case
So now what sbuild does is this:
dpkg-buildpackage --target clean
This takes care of all of the above by only selectively running fakeroot and
by checking for missing build dependencies.
You have several options to work around this:
a) don't run the clean target. Either by running sbuild with --no-clean or
by putting $clean_source=0; into your ~/.config/sbuild/config.pl Doing
this is especially useful if you are running sbuild from a packaging
git and you thus already know that your source is clean, either becaus
you ran "git clean -fdx" yourself or because git-buildpackage stops you
from building a package in a not-clean state.
b) Avoid putting packages in Build-Depends. I've had a look at your package
kde-spectacle and I'm quite sure that you do not need all these *-dev
packages to run the clean target. This patch would fix it for you:
--- a/debian/control
+++ b/debian/control
@@ -3,8 +3,8 @@ Section: kde
Priority: optional
Maintainer: Debian Qt/KDE Maintainers <debian-qt-kde@lists.debian.org>
Uploaders: Aurélien COUDERC <coucouf@debian.org>,
-Build-Depends: cmake (>= 3.16~),
- debhelper-compat (= 13),
+Build-Depends: debhelper-compat (= 13),
+Build-Depends-Arch: cmake (>= 3.16~),
dh-sequence-kf6,
dh-sequence-qmldeps,
extra-cmake-modules (>= 6.3.0~),
Have a look at Debian policy §7.7. To run the clean target, only B-D
and B-C must be satisfied. So you can move what is actually needed
for the build-arch target into B-D-A instead.
c) Convince me that sbuild should instead run the clean target of the
packages it builds without making sure that the required dependencies
to do so are satisfied.
d) Convince me that disabling running the clean target by default is a good
idea. There is this MR for that:
https://salsa.debian.org/debian/sbuild/-/merge_requests/71
Thanks!
cheers, josch
[+pkg-kde-talk for ideas / comments] Le mardi 26 novembre 2024, 10:39:47 UTC+1 Johannes Schauer Marin Rodrigues a écrit : Thanks for the detailed explanation. I did see the change in the changelog but couldn’t relate it to the error I was seeing. If I, building loads of packages every other day, cannot make up what needs to be done to get it working how will newcomers or occasional contributors get around that ? For me sbuild should work from the source package without additional setup, as much as sensible. b) Doable. Lot of work. Need to change our somewhat complex tooling [1] that generates and updates the build deps from upstream’s CMakeFiles. Also (maybe biased because we package mostly arch-dependent binaries) I would seem more logical to me to have a Build-Depends-Noarch stanza with the very few package needed so that the default Build-Depends contains the useful stuff. c) Looks like a recipe for failure. Maybe only for relatively corner cases but it looks unexpected. d) If that ends up adding unexpected files to the build, clearly not a good option either. I don’t pretend that my build environments are clean enough that I can afford that. So maybe that’s a valid option but I’m not the one going to try and convince you. :) You’re talking about fakeroot but is that the interesting part ? It seems to me that sbuild previously ran the clean target inside the container having B-D installed, which is why it was working. I think that’s a property worth keeping regardless of the use of fakeroot. So my preferred solution would be to mimic that behaviour and have : - the source tree is injected in the build container with the B-D installed, - the clean and build source stages happen there, - the container is cleaned / reverted to its initial state with only the B-D, - then it’s reused for the actual build. [1] https://salsa.debian.org/qt-kde-team/pkg-kde-dev-scripts/-/blob/master/function_collection/cmake_update_deps.py?ref_type=heads Best, -- Aurélien
Hi,
Quoting Aurélien COUDERC (2024-11-26 21:35:38)
that is precisely an arguement for d). If packaging is done in git and if the
git clone is clean, then what is the point of attempting to run the clean
target?
Historically, we first had Build-{Depends,Conflicts}-Indep because people felt
it would be useful to single out the build dependencies that are needed for
building the architecture independent packages such that the arch:all buildds
do not have to install potentially insane amounts of dependencies to only
assemble a few arch:all packages. Then later, the sake of symmetry, the
Build-{Depends,Conflicts}-Arch fields were added to build arch:any packages.
See bug #629480 if you want to dive into the history of that. It's probably
much easier to change your tooling than to add a new field.
I agree. And that's why I think that the change is good and here to stay.
My main argument against changing the default is, that it is just very easy to
change the default of this setting on your machine. If we were to change the
default, this would definitely warrant an entry in NEWS because people are
likely relying on this.
No, fakeroot is not the interesting part here. Avoiding fakeroot is just
amongst the reasons why that change was performed. The other reason being that
before, we ran the clean target without checking that the dependencies to do so
were installed.
This is not changing. Sbuild still runs the clean target inside the chroot with
all B-Ds installed.
The reason you notice the fact that sbuild runs the clean target on the outside
as well is, that now it also checks whether the dependencies required to run
the clean target are also installed. It did not do that before. But since
effectively all you need to run the clean target is debhelper and since you
probably have debhelper installed on the outside, you never noticed that you
did not have the build dependencies installed.
Doing this is certainly possible but it would require a large refactoring of
how sbuild operates. Currently, the way how source packages are transported
into the chroot are via a dsc and the files it references. If you call sbuild
on an unpacked source tree, then the dsc has to be built first. To avoid
surprises, sbuild calls the clean target before running "dpkg-source -b ."
Changing the way source packages are transported into the chroot from dsc to
something else would be a very large effort. But of course, patches are
welcome. But this gets me to an option I forgot to list earlier:
e) do not run sbuild inside an unpacked source directory. Instead, create the
dsc in any way you like and then feed that dsc file as a positional
argument to sbuild. But this option is not so interesting I think because
if you create the dsc manually, then you have to verify that your unpacked
source is clean and if you have done that, then you can call sbuild with
--no-clean already and call it a day...
Thanks!
cheers, josch
Dear Maintainer, I am also suffering from this bug. Downgrading sbuild to 0.87.0 from testing fixes it. ``` gbp buildpackage -v --source-on ly gbp:info: Performing the build dpkg-buildpackage: info: source package golang-github-protonmail-gopenpgp-v3 dpkg-buildpackage: info: source version 3.1.0-2 dpkg-buildpackage: info: source distribution unstable dpkg-buildpackage: info: source changed by Martin Dosch <martin@mdosch.de> dpkg-checkbuilddeps: error: Unmet build dependencies: golang-github-pkg-errors-dev golang-github-protonmail-go-crypto-dev (>= 1.1.0~) golang-github-protonmail-go-mime-dev golang-github-stretchr-testify-dev dpkg-buildpackage: warning: build dependencies/conflicts unsatisfied; aborting dpkg-buildpackage: warning: (Use -d flag to override.) E: Failed to clean source directory /home/martin/build/deb/golang-packaging/golang-github-protonmail-gopenpgp-v3 (/home/martin/build/deb/golang-packaging/golang-github-protonmail-gopenpgp-v3_3.1.0-2.dsc) gbp:error: 'sbuild -v --source-only' failed: it exited with 1 ``` So to workaround this bug I have to either disable clean up or not use build-depends, do I understand this correct? Maybe add `$clean_source=0;` by default to not break build systems with the update? Best regards, Martin
Hi, Quoting Martin Dosch (2024-11-27 19:23:57) this is not a bug. What you are reporting is the result of a bug being fixed. It doesn't fix it. It hides a problem. What downgrading hides is, that running the clean target is done without checking whether the packages necessary to run the clean target are installed. That's a bug. You are using gbp buildpackage around sbuild. Unless you configured it otherwise, gbp buildpackage will not let you proceed if your git clone is not clean. So before running gbp buildpackage you have to clean up the unpacked source, for example by running "git clean -fdx". It is thus indeed quite pointless to give the task of cleaning the unpacked source to sbuild. If you run sbuild via gbp, you should probably configure gbp such that it always passes --no-clean to sbuild. If the default for $clean_source is changed, then that will break things for those people who relied on sbuild performing the clean step. Changing the default is an option, but it is one that has to be done very carefully. Whether or not the default for cleaning the source should be changed can be discussed here: https://salsa.debian.org/debian/sbuild/-/merge_requests/71 I have yet to hear an argument that convinces me. Not cleaning the source by default is *not* the fix to your problem though. The situation you had before was the problem. You just didn't see it because you probably had debhelper and some other packages installed. For example, you are currently working on src:golang-github-protonmail-gopenpgp-v3. If I just run "debian/rules clean" on my system I get this: dh clean --builddirectory=_build --buildsystem=golang dh: error: unable to load addon golang: Can't locate Debian/Debhelper/Sequence/golang.pm in @INC (you may need to install the Debian::Debhelper::Sequence::golang module) (@INC contains: /etc/perl /usr/local/lib/aarch64-linux-gnu/perl/5.36.0 /usr/local/share/perl/5.36.0 /usr/lib/aarch64-linux-gnu/perl5/5.36 /usr/share/perl5 /usr/lib/aarch64-linux-gnu/perl-base /usr/lib/aarch64-linux-gnu/perl/5.36 /usr/share/perl/5.36 /usr/local/lib/site_perl) at (eval 13) line 1. BEGIN failed--compilation aborted at (eval 13) line 1. This happened because I did not have the dependencies required to run the clean target of this package installed. Sbuild not complaining about this before this error happens is a bug. So now, to fix the bug, sbuild checks that the packages required to run the clean target are installed before running it. Thanks! cheers, josch
Dear Johannes, thank you very much for the detailled clarification. Best regards, Martin