Hello!
Trying to cross-compile the ghc package using "sbuild --host=$ARCH" now fails
with:
dh_auto_configure -- \
--target sh4-linux-gnu --enable-unregisterised --with-ghc="/usr/bin/ghc" \
--with-system-libffi --libdir=/usr/lib
./configure --build=x86_64-linux-gnu --prefix=/usr --includedir=\${prefix}/include --mandir=\${prefix}/share/man --infodir=\${prefix}/share/info --sysconfdir=/etc --localstatedir=/var --disable-silent-rules --libdir=\${prefix}/lib/sh4-linux-gnu --libexecdir=\${prefix}/lib/sh4-linux-gnu --disable-maintainer-mode --disable-dependency-tracking --host=sh4-linux-gnu --target sh4-linux-gnu --enable-unregisterised --with-ghc=/usr/bin/ghc --with-system-libffi --libdir=/usr/lib
configure: WARNING: unrecognized options: --disable-silent-rules, --disable-maintainer-mode, --disable-dependency-tracking
configure: loading site script /etc/dpkg-cross/cross-config.sh4
/etc/dpkg-cross/cross-config.sh: line 49: syntax error near unexpected token `fi'
/etc/dpkg-cross/cross-config.sh: line 49: `fi'
checking for gfind... no
checking for find... /usr/bin/find
checking for sort... /usr/bin/sort
checking for GHC Git commit id... given 51abb1c88b53e2989a2a8c2939ac4abc04bef194
checking version of ghc... 8.4.3
checking build system type... x86_64-pc-linux-gnu
checking host system type... sh4-unknown-linux-gnu
checking target system type... sh4-unknown-linux-gnu
GHC build : x86_64-unknown-linux
GHC host : sh4-unknown-linux
GHC target : sh4-unknown-linux
LLVM target: sh4-unknown-linux
checking for path to top of build tree... /<<BUILDDIR>>/ghc-8.4.3+dfsg1
configure: error:
You've selected:
BUILD: x86_64-unknown-linux (the architecture we're building on)
HOST: sh4-unknown-linux (the architecture the compiler we're building will execute on)
TARGET: sh4-unknown-linux (the architecture the compiler we're building will produce code for)
BUILD must equal HOST; that is, we do not support building GHC itself
with a cross-compiler. To cross-compile GHC itself, set TARGET: stage
1 will be a cross-compiler, and stage 2 will be the cross-compiled
GHC.
From the error message it looks like, we need to strip "--host=$ARCH" from the
configure line and just use "--target=$ARCH" for that.
Thanks,
Adrian
Hi! This change to the debian/rules file helps to get a bit further:--- debian/rules.orig 2018-09-26 11:08:46.000000000 +0200 +++ debian/rules 2018-10-09 15:22:43.080942145 +0200 @@ -126,6 +126,7 @@ echo 'V=1' >> mk/build.mk dh_auto_configure -- \ $(EXTRA_CONFIGURE_FLAGS) \ + --host=$(DEB_BUILD_GNU_TYPE) \ --with-system-libffi --libdir=/usr/lib override_dh_auto_build: But it will still fail with: ===--- building phase 0 make --no-print-directory -f ghc.mk phase=0 phase_0_builds mkdir -p inplace/bin mkdir -p inplace/lib "rm" -f libraries/binary/dist-boot/build/libHS.a libraries/binary/dist-boot/build/libHS.a.contents "rm" -f libraries/text/dist-boot/build/libHS.a libraries/text/dist-boot/build/libHS.a.contents "rm" -f libraries/transformers/dist-boot/build/libHS.a libraries/transformers/dist-boot/build/libHS.a.contents "rm" -f libraries/mtl/dist-boot/build/libHS.a libraries/mtl/dist-boot/build/libHS.a.contents "rm" -f inplace/bin/mkdirhier "rm" -f libraries/parsec/dist-boot/build/libHS.a libraries/parsec/dist-boot/build/libHS.a.contents echo >> libraries/binary/dist-boot/build/libHS.a.contents "rm" -f libraries/Cabal/Cabal/dist-boot/build/libHS.a libraries/Cabal/Cabal/dist-boot/build/libHS.a.contents echo >> libraries/text/dist-boot/build/libHS.a.contents echo >> libraries/transformers/dist-boot/build/libHS.a.contents /bin/bash: libraries/binary/dist-boot/build/libHS.a.contents: No such file or directory echo >> libraries/mtl/dist-boot/build/libHS.a.contents echo '#!/bin/sh' >> inplace/bin/mkdirhier /bin/bash: libraries/text/dist-boot/build/libHS.a.contents: No such file or directory echo >> libraries/parsec/dist-boot/build/libHS.a.contents utils/hsc2hs/ghc.mk:24: utils/hsc2hs/dist-install/package-data.mk: No such file or directory make[3]: *** [libraries/binary/ghc.mk:4: libraries/binary/dist-boot/build/libHS.a] Error 1 make[3]: *** Waiting for unfinished jobs.... make[3]: *** [libraries/text/ghc.mk:4: libraries/text/dist-boot/build/libHS.a] Error 1 /bin/bash: libraries/mtl/dist-boot/build/libHS.a.contents: No such file or directory /bin/bash: libraries/transformers/dist-boot/build/libHS.a.contents: No such file or directory echo >> libraries/Cabal/Cabal/dist-boot/build/libHS.a.contents make[3]: *** [libraries/transformers/ghc.mk:4: libraries/transformers/dist-boot/build/libHS.a] Error 1 make[3]: *** [libraries/mtl/ghc.mk:4: libraries/mtl/dist-boot/build/libHS.a] Error 1 cat utils/mkdirhier/mkdirhier.sh >> inplace/bin/mkdirhier /bin/bash: libraries/parsec/dist-boot/build/libHS.a.contents: No such file or directory make[3]: *** [libraries/parsec/ghc.mk:4: libraries/parsec/dist-boot/build/libHS.a] Error 1 /bin/bash: libraries/Cabal/Cabal/dist-boot/build/libHS.a.contents: No such file or directory make[3]: *** [libraries/Cabal/Cabal/ghc.mk:4: libraries/Cabal/Cabal/dist-boot/build/libHS.a] Error 1 chmod +x inplace/bin/mkdirhier make[2]: *** [Makefile:123: all] Error 2 make[2]: Leaving directory '/<<BUILDDIR>>/ghc-8.4.3+dfsg1' dh_auto_build: make -j8 returned exit code 2 make[1]: *** [debian/rules:133: override_dh_auto_build] Error 2 make[1]: Leaving directory '/<<BUILDDIR>>/ghc-8.4.3+dfsg1' make: *** [debian/rules:58: binary-arch] Error 2 dpkg-buildpackage: error: debian/rules binary-arch subprocess returned exit status 2 Will try to figure out what's wrong here. Adrian
user debian-cross@lists.debian.org usertags 910664 + ftcbfs tags 910664 + patch thanks Hi, Given that the earlier bug about haskell-devscripts-minimal has been fixed, I happen to have looked into this. First and foremost, ghc now actively refuses being cross build with an $(error ...). Would you mind weakening this to a $(warning ...)? While that you don't want to support cross building of ghc, would you mind others (like me) supporting it? Yes, it would still fail, but then https://crossqa.debian.net/src/ghc could show a more useful reason for that failure. (patch attached) Then when you get past the $(error ...), stage1 tools are not found. In a cross build, ghc prefixes these with the host gnu triplet. A bit of renaming is required to make this work. (patch attached) And then we run into: I think this is wrong. The current ghc packaging includes this flag, but ghc uses the same terminology as Debian, so when we say --host=$(DEB_BUILD_GNU_TYPE) we ask it to produce a cross compiler, but we really wanted a cross built ghc. One of the issues referenced from the earlier $(error ...) also hints that this value is wrong for --host: https://gitlab.haskell.org/ghc/ghc/-/issues/22006 Unless you have strong reasons for why that should be correct, I suggest that we change it to --host=$(DEB_HOST_GNU_TYPE). (not included in patch) So this is where we are. I think we can make progress here if you want to support this work. I'm also a big fan of actionable bug reports and in having a patch, this bug becomes actionable. Given that you (ghc maintainers) are evidently not interested in doing the work here, I also suggest that you close this bug when applying the patch and letting cross users file new bugs with new patches. Let me know what you think about this. Helmut
Hi Helmut, Thanks for offering to help with cross building. Let me first comment on the following. and The above statements are not true. While rewriting d/rules to build GHC using Hadrian, I tried very hard to implement cross-building for GHC. I backported upstream fixes [1], engaged in upstream discussion about this [2] and we concluded that the new Hadrian build system is not capable of cross-building GHC [3]. Upstream is working on fixing this, but it's a slow process [4]. This is why we have the error message there, and if you look at the d/rules file, you will see it has comments that link to these upstream issues. With that said, I am happy to share everything I have seen while trying to make cross-builds work, and we can work together to support this. [1] https://salsa.debian.org/haskell-team/DHG_packages/-/blob/5e08edfeba1729c62d7292cdf47408de95a5b37c/p/ghc/debian/patches/fix-cross-compilation [2] https://gitlab.haskell.org/ghc/ghc/-/issues/23975 [3] https://gitlab.haskell.org/ghc/ghc/-/issues/23975#note_526546 [4] https://gitlab.haskell.org/ghc/ghc/-/issues/23975#note_530201 Best,
Hi Helmut, The reason I have this as an $(error ...) is because the new Hadrian build system doesn't support cross-compiling GHC [1]. This is not a limitation of our Debianization (i.e., it's not that we refuse to cross-build GHC, we *cannot* cross-build GHC), this is upstream limitation. Given that, I believe $(error ...) is more appropriate here than $(warning ...). [1] https://gitlab.haskell.org/ghc/ghc/-/issues/23975 The resulting stage1 tools are a cross-compiler, not the final cross-compiled binaries (this is why they are prefixed with the host gnu triplet). I am not sure how we will handle this, I am waiting to see on what upstream will do to support cross-building GHC. For now, I believe applying the attached patch doesn't help. GHC does *not* use the same terminology as Debian. GHC requires that HOST and BUILD are the same. You can read more about this here [2], though keep in mind this document is severely outdated with the Hadrian build system. [2] https://gitlab.haskell.org/ghc/ghc/-/wikis/building/cross-compiling#terminology-and-background I believe we need to work with upstream to add support for cross-compiling GHC to the new Hadrian build system. As explained here [3], this is currently not possible. I tested everything I could think of, but I don't see how we can move forward without reworking how the Hadrian build system works. Upstream has started working on this, but it's moving slowly [4]. In summary, as a result of the switch to the Hadrian build system, we are now unable to cross-compile GHC. Since this issue now has all the latest context, I propose we keep it open and work here on cross-building GHC. [3] https://gitlab.haskell.org/ghc/ghc/-/issues/23975#note_526546 [4] https://gitlab.haskell.org/ghc/ghc/-/issues/23975#note_530201 Thanks,
Hi Ilias,
I'm not yet entirely convinced that it cannot work, but I certainly
agree that it does not work now. However, the $(error ...) completely
hides the reason. I'd like to see a cross build failure that shows why
this cannot work and that failure only comes about much later. What
advantage do you see in hiding this failure from QA systems?
To me, this looks fairly obvious. The stage1 tools are not installed
into any .deb at any time. What we install into a .deb is the stage2.
All that we need stage1 for is running it during build. This is where my
patch helps. And quite objectively, the patch helps in the sense that
the build goes further once you apply it.
That page carefully explains that it uses the same terminology as
Debian. It also says that host and build must equal which translates to
"cross compilation is not supported at all", because the definition of a
cross build is build and host being different from one another.
However, that page also goes on and says that stage2 is being built
using stage1. Since stage1 is a cross compiler, stage1 runs on the build
machine and hence we are building stage2 on our build machine. And since
stage2 runs on the target, the stage2 build process effectively has
build differ from host contradicting what the page says earlier. So
indeed, something is wrong here. From my pov, the information that most
likely is wrong is that ghc cannot be cross compiled.
I agree that we need to work with upstream. However, we also should make
it work on the Debian side as good as possible, but currently it aborts
early saying that this cannot work. Can we move incrementally and enable
as much as works?
The Debian build does not get far enough to demonstrate this failure.
Talking to upstream is much easier if you can point them at failing
build logs and we currently cannot, because we are still facing issues
before even running Hadrian.
Regarding that --build flag, I now see that we actually run configure
twice. Connecting what you said earlier, I guess that one of those
invocations is for the stage1 compiler and the second one is for the
stage2 compiler though I may be wrong. If that guess is correct, then
yes, the stage1 compiler definitely needs --host=$(DEB_BUILD_GNU_TYPE),
but doing it like that still passes host architecture compiler flags and
you end up running e.g. aarch64-linux-gnu-gcc -fcf-protection or
x86_64-linux-gnu-gcc -mbranch-protection=standard and fail either way.
So the way to pass that --host flag is:
dpkg-architecture -a$(DEB_BUILD_ARCH) -f -c dh_auto_configure --reload-all-buildenv-variables -- $(CONFIGURE_ARGS)
# CONFIGURE_ARGS contains neither --build nor --host, but it contains --target
Then when we build stage2, we are using stage1 and there we probably
should not be passing --host=$(DEB_BUILD_GNU_TYPE). My guess now is that
this is where that wiki page is wrong.
Given all of the above, I still think that my patch moves us forward,
because it allows https://crossqa.debian.net/src/ghc display a real
failure that we can use in communication with upstream and because
calling stage1 tools in the right way pretty much is required to seeing
the later failure from using Hadrian and being able to work on that
aspect. This failure is not very useful to look at:
https://crossqa.debian.net/build/ghc_9.4.7-1_amd64_20231029134737.log
Is this reasoning good enough to convince you that we do have options
for improving the status quo?
Helmut
Hi Helmut, Sorry for the late reply. Comments inline. Let me try to explain how GHC's build system works with the new Hadrian tool. Then let's see how we can move forward and make it so that the build fails at the right stage. 1. Run boot and configure. Here is where we decide if we want to build GHC for the same architecture or do cross compilation. 2. Run hadrian. The build artifacts will be under '_build/stageX'. With the Hadrian build system, the naming convention for these directories have changed. '_build/stage1' contains the binaries that were built using the stage1 compiler. If we are building a compiler for the same architecture, '_build/stage1/' will contain the stage2 compiler for this architecture (i.e., the binaries that we ship in our Debian package). If we are cross compiling, '_build/stage1/' will contain a *stage1 cross-compiler*. As discussed in this issue [1], upstream is working towards making this a *stage2 cross-compiler*. Is is still a cross-compiler though, and not a cross-compiled GHC as it used to be. The final output of this step is a binary distribution of GHC, i.e., a tarball that you can distribute to anyone that wants to install GHC. 3. We run configure and 'make install' to convert the binary distribution to a Debian binary package. This is why we run configure twice. [1] https://gitlab.haskell.org/ghc/ghc/-/issues/19174 I hope the above explains why we cannot cross compile GHC (the build system doesn't give us this option). Given the above, I believe your patch needs a few adjustments. As explained, this is not the stage1 tool, this is the tool built by the stage1 compiler, and this directory contains vastly different tools depending on whether we cross-compile or not. In case we are cross-compiling, this is the cross-compiler ghc-pkg. Calling 'ghc-pkg recache' here is wrong. I suppose we can skip this step if we are cross-compiling (so we can reach the next failure). Running the tests when cross-compiling will not work. Either we are calling the cross-compiler (which will produce binaries we cannot test) or we are calling a cross-compiled GHC which will not work. I propose we skip the tests as well, and let the build fail elsewhere. I don't object in removing the error if this will help us move forward. I can also apply your patch with the above changes (i.e., skip some steps, instead of calling the cross-compiler). But I really believe we should focus on having end-to-end instructions on how to cross-compile GHC and then try to fix Debian's build rules. Fixing errors as we see them doesn't always move us towards the right direction, if we don't understand how the end-to-end process works.
Hi Ilias, Thank you for taking the time to write such a detailed response. I fear this is misleadingly imprecise. If I understand what you write later correctly, we always build GHC for the same architecture (i.e. we ask configure for build=host), but we may opt for a cross compiler (i.e. target!=host). When the ghc Debian package is asked to cross compile, what we ask configure instead is natively building a cross compiler. Do you confirm? This is still quite confusing to me. I suppose stage0 is /usr/bin/ghc. Do you confirm? Then there should be a _build/stage0 containing the stage1 compiler. That stage1 compiler probably is a simple native build of the ghc sources at hand, which means that its binary may have a different ABI from what the sources expect. That stage1 will be unable to use packages, but it can be used to build ghc again. Is that also correct? Then that stage1 is used to build a stage2 where the ABI of the binary matches the behaviour and this can be used with packages. This would be consistent with the possible understanding given above. Do I understand correctly that (currently) stage1 is a simple native build of the current ghc sources where the binary uses the ABI that /usr/bin/ghc generated and doesn't necessarily match the sources? Do I also understand that stage2 (stored in _build/stage1) the is a cross compiler generating code for $DEB_HOST_ARCH and runnable on $DEB_BUILD_ARCH using the ABI given by the current ghc sources? At no point do we (currently) actually cross build using that stage2. Do you also confirm? I fear it does not. It explains why we currently do not cross compile GHC, but I still fail to understand why we cannot. (Or maybe this is splitting hairs and we mean the same thing with different words.) Probably. As far as I can see, the name "STAGE1_TOOL" is misleading and it should be called "STAGE2_TOOL" instead. Do you concur? Then a STAGE2_TOOL always is something that runs on the build machine and operates on $DEB_HOST_ARCH, which seems just about right, no? STAGE2_TOOL not necessarily is something we'd want to install into a .deb though. Do you agree? This is how I expected it, yes. Can you elaborate on why we do not want to reset the package cache here? Most of the time, this is true. However, we can also cross build from amd64 to i386 or arm64 to armhf (which gets us 64bit address space during build) and we can build with a qemu-user-static installed. In both cases, we can actually run tests. Therefore the decision whether to run tests is left to the builder. Both sbuild and pbuilder default to not running tests by automatically adding nocheck to DEB_BUILD_OPTIONS when you ask for a cross build. This whole block is conditional to DEB_BUILD_OPTIONS not containing nocheck, so the only way this is relevant is when a builder overrides this default and thus explicitly requests running tests despite performing a cross build. And in that case, the proposed patch should make sense, no? Your proposed skipping already is implemented via nocheck. Thank you. Thank you for considering "my way". You have made a good case for understanding the end-to-end process and you definitely convinced me that this is necessary. Often times, the incremental process just works and here it likely is not ideal, but the discussion still seems to advance us and I would be more than happy to continue and improve our understanding to reach that state where we make that end-to-end process work practically. Hope you can bear with me. Please take your time to respond even if that happens to be next year. This is not something we have to fix right now. I prefer a good and maintainable solution over a quick solution. Helmut
Hi Helmut, Happy new year! Yes I confirm. Up until now, this was the way to get both a cross compiler (the STAGE1_TOOL compiler) and a cross-compiled GHC (the STAGE2_TOOL compiler) [1], and we installed in the Debian package the STAGE2_TOOL compiler. [1] https://gitlab.haskell.org/ghc/ghc/-/wikis/building/cross-compiling#terminology-and-background This changed with the new build-system, and now we get only a cross compiler both from stage1 and stage2. One can of course argue that this is the expected output (given we passed build=host) :) Yes on all of the above, this is my understanding as well. No we don't. I have tried doing that by: 1. Trying to build stage 3, but the resulting compiler was still a cross compiler, see [2]. 2. Trying to use the resulting stage 2 compiler as a bootstrap compiler (stage 0) for a new build, see [3]. Both of the above approaches failed. This is where we are currently blocked, and I admit I haven't tried to move past this. [2] https://gitlab.haskell.org/ghc/ghc/-/issues/23975#note_526549 [3] https://gitlab.haskell.org/ghc/ghc/-/issues/23975#note_530085 Yes I agree. What we want to do here is to be able to run ghc-pkg and query the GHC database for the library ABIs. But this needs to be done for the HOST/TARGET architecture. I am not really sure if running the STAGE2_TOOL ghc-pkg here (which runs on the BUILD architecture) will produce the same ABIs as the STAGE3_TOOL ghc-pkg (if we had one). You are right, sbuild by default will pass DEB_BUILD_OPTIONS=nocheck so no need to explicitly disable our tests. And if one explicitly asks for them, we can definitely find a way to support this. Thank you for helping here. This discussion really helps move things forward. I have now uploaded GHC 9.6.4-1~exp1 to experimental where I disabled the explicit error and let it fail at a later step. I didn't fix the tests for now, let me know if this causes problems with our QA system.