Package: apt Version: 0.8.16~exp12 Severity: normal Tags: patch User: multiarch-devel@lists.alioth.debian.org Usertags: multiarch Hey guys, While working on getting the Ubuntu base system's cross-build-dependencies satisfiable using only apt-get build-dep -a $arch, I came to the realization that the only Architecture: all package I'd seen being used as a build-dependency that *couldn't* be marked Multi-Arch: foreign was python-dev - and we had separately determined that python-dev needs to be made an Architecture: any package precisely *because* other packages needed to build-depend on the host version of python2.7-dev for cross-building. And of course, unlike in the case of binary deps, there's no way that the new build-dep behavior would accidentally pull in the wrong dependencies and break a user's system on upgrade. In both cases, an Architecture: all package that masks a transitive arch-dependent dependency is *wrong*; but whereas in the binary dep case we have to be very conservative in treating that package the same way previous package managers would have (i.e., treating it like a native package), cross-build-dep handling is an entirely new feature with no upgrade considerations - so it should be entirely reasonable to treat all Arch: all packages as implicitly M-A: foreign when satisfying cross-build-deps. In support of this argument I'm attaching two bits of evidence: - arch-indep-build-deps.sh, that recurses through all the build-deps in the specified distro (here, Ubuntu) to generate a list of Arch: all packages that are used as build-dependencies. - arch-all-build-dep-list.txt, the sorted output of the above script. This shows the scope of the problem; up to 1000 Arch: all packages that might need to be marked M-A: foreign without this change, which may not need any metadata updates with this change. (Of course, this script doesn't filter out build-deps of source packages that only build Architecture: all binaries, so the actual number of packages affected may be somewhat lower.) Finally, I'm attaching a patch that I believe implements the desired behavior. I've tested this locally, and it does the right thing in my tests. Example: Before: $ sudo apt-get build-dep -a armhf htmldoc Reading package lists... Done Building dependency tree Reading state information... Done Note, selecting 'libjpeg-turbo8-dev:armhf' instead of 'libjpeg-dev:armhf' Note, selecting 'libpng12-dev' instead of 'libpng-dev' E: Build-Depends dependency for htmldoc cannot be satisfied because the package autoconf2.59 cannot be found $ After: $ sudo apt-get build-dep -a armhf htmldoc Reading package lists... Done Building dependency tree Reading state information... Done Note, selecting 'libpng12-dev' instead of 'libpng-dev' The following NEW packages will be installed: autoconf2.59 libdrm2:armhf libexpat1:armhf libfltk1.1:armhf libfltk1.1-dev:armhf libfontconfig1:armhf libfreetype6:armhf libgl1-mesa-glx:armhf libglapi-mesa:armhf libjpeg-dev libjpeg-turbo8:armhf libjpeg-turbo8-dev libjpeg8:armhf libjpeg8-dev libpng12-0:armhf libpng12-dev libssl-dev:armhf libssl1.0.0:armhf libx11-xcb1:armhf libxcb-glx0:armhf libxdamage1:armhf libxfixes3:armhf libxft2:armhf libxinerama1:armhf libxrender1:armhf libxxf86vm1:armhf zlib1g:armhf zlib1g-dev zlib1g-dev:armhf The following packages will be upgraded: libssl1.0.0 1 upgraded, 29 newly installed, 0 to remove and 33 not upgraded. Need to get 2134 kB/6199 kB of archives. After this operation, 15.6 MB of additional disk space will be used. Do you want to continue [Y/n]? ^C $ Of note here is that libjpeg-dev is an Arch: all transitional package depending on an Arch: any libjpeg8-dev, so apt doesn't get the answer completely right - but as mentioned, the only way to fix that case is by making libjpeg-dev not be Arch: all. Thanks,
Refined patch, now with test cases included. This patch has been uploaded to Ubuntu for precise. Thanks,
Have you talked to the dpkg developers about this? There is at least dpkg-checkbuilddeps which checks build dependencies, and I think dpkg also does build-time dependency handling in some other tools. And as dpkg is one level lower than APT, I think they should be asked first before doing something like this.
Hi, TL;DR Dpkg does the useful thing, apt-get build-dep doesn't. I've just been getting a setup working for Debian cross-building. At work I use OpenEmbedded which is a time consuming pain in the neck. I hope to get to a work flow that can replace crappy embedded vendor toolchains but for the moment I'm following Wookie's recipe at https://wiki.debian.org/CrossToolchains which complements his talk at Cambridge mini-debconf. To the extent that there is any maintainer-supported user-level cross building support in Debian, this seems to be it from my reading. However this bug is a significant blocker to that work process. Several of the basic Priority Essential packages won't cross-build with target != host because of the current bug. dpkg-checkbuilddeps already allows Arch:all Multi-Arch:none packages to satisfy build-depends. I've built apt with the patch here and this allows "apt-get build-dep" to do the useful thing also. Both the failures in the attached log go away with this patch to apt. Please therefore can this patch be applied and released ? Many thanks Nick Leverton
Err s/Wookie/Wookey/ *ahem* sorry. Nick
Hi, Quoting Nick Leverton (2014-11-21 01:50:54) I don't think dpkg behaves different than apt here. Can you give the example? Maybe you are running into dpkg bug #710769? Treating Architecture:all packages implicitly as Multi-Arch:foreign would be wrong for many Architecture:all packages. Here two examples of Architecture:all packages that can NOT be marked Multi-Arch:foreign (without further work): - haskell-devscripts #769377 - dh-ocaml #769379 If apt would treat all Architecture:all packages as Multi-Arch:foreign then these two packages would have a wrong Multi-Arch annotation. This shows that marking packages as Multi-Arch:foreign must be done with care and not in an automated fashion. cheers, josch
Dear colleagues, Nine years after this bug had last activity upon I hit it :) I was trying to cross-compile Kodi for armhf on amd64 and noticed that apt-get treats arch:all build-dependencies as arch:native and fails to find them. Furthermore, packages providing the implementation of virtual packages from Build-Depends list (aka "Reverse Provides") are treated the same way. Since Wookey vetted Steve's apt-get patch I decided to work around this bug in pbuilder-satisfydepends-apt and filed the MR#20 on Salsa: https://salsa.debian.org/pbuilder-team/pbuilder/-/merge_requests/20 I tested that one building Kodi, its addons and several other random packages. -- Vasyl Gello ================================================== Certified SolidWorks Expert Mob.:+380 (98) 465 66 77 E-Mail: vasek.gello@gmail.com Skype: vasek.gello ================================================== 호랑이는 죽어서 가죽을 남기고 사람은 죽어서 이름을 남긴다
Quoting Vasyl Gello (2021-10-28 16:10:38) Good. That behaviour is by design and not a bug. Good. This is not a bug. Architecture:all packages are *not* implicitly M-A:foreign. See my other mail to this bug. If an Architecture:all package should be made M-A:foreign, then file a bug against that package. The maintainer can then decide whether that marking is correct or not. If in doubt, CC debian-cross@lists.debian.org to get advice about whether marking a package as M-A:foreign is correct or not. Thanks! cheers, josch
Hi Johannes! Thanks for clarification! So I have to ask maintainers of groovy:all and libcommons-lang-java:all whether they are OK marking it M-A: foreign? Also, why does apt-get install all arch:host build-deps but not libdevel:host duplicated with libdevel:build ? Is it possible at all to have "pure" cross-compilation without need to install qemu-binfmt-support on host? -- Vasyl Gello ================================================== Certified SolidWorks Expert Mob.:+380 (98) 465 66 77 E-Mail: vasek.gello@gmail.com Skype: vasek.gello ================================================== 호랑이는 죽어서 가죽을 남기고 사람은 죽어서 이름을 남긴다