libnet-dbus-perl fails to cross build from source, because its perl
(host) dependency is unsatisfiable. Based on Niko's work, we can update
it to perl-xs-dev and this is where it becomes interesting:

libnet-dbus-perl fails to find the dbus library. It does so using the
build architecture pkg-config, but libdbus is only installed for the
host architecture. Now I'm unsure how to fix this. The common wisdom is
to prefix tools with the host architecture triplet. The other wisdom is
to pass tools as environment variables. The perl ecosystem seems to
follow a different route: Record paths in Config.pm. Unfortunately,
Config.pm neither has the triplet nor does it have pkg-config. Bummer.
So what is a good way to get the right pkg-config into the Makefile.PL?

I'm attaching a patch to demonstrate that pkg-config really is the only
issue: It makes libnet-pkg-config cross buildable by passing pkg-config
through the environment. Please consider it RFC and comment on it rather
than applying it (which also is why I don't tag the bug with patch).

I don't like the patch, because it creates two routes to communicate the
host architecture: Config.pm and environment. If these communication
methods are not in sync, bad things happen. I'd much prefer to have a
single source of truth here.

Consider this bug an "example bug". Using pkg-config in Makefile.PL is
likely something that happens elsewhere. It'll need fixing a few times.
I'd like to get to a good solution here before filing more of them,
because libnet-dbus-perl nicely isolates this issue.

You can apply the perl-xs-dev part right away if you like, but it isn't
sufficient.

Helmut

Thanks for the bug report, that's an interesting one indeed.

So we're looking for something like 'x86_64-linux-gnu' in Config.pm
(or pass PKG_CONFIG=x86_64-linux-gnu-pkg-config in the environment),
right?

For the former,
    % perl -MConfig=config_sh -e 'print config_sh();' | grep x86_64-linux-gnu
has quite a few results but never x86_64-linux-gnu alone.

(And Build.PL.)
I guess that's a question for Niko …

Ack.

Right, but the good news is that I only find 23 packages (of the
pkg-perl maintained ones) which use pkg-config in
{Makefile,Build}.PL.


Random thought: I wonder if PkgConfig ("a pure perl version of
pkg-config"), packaged as libpkgconfig-perl, providing
/usr/bin/ppkg-config, could be a helpful workaround. This would still
require a changed build dependency and a s/pkg-config/ppkg-config/ in
{Makefile,Build}.PL [0] but would spare us any path or architecture
hassles. Maybe :)


*Some times later*


So, I did my first cross build :) (with cowbuilder and the help of
https://wiki.debian.org/CrossCompiling ) and it worked with:

+               libpkgconfig-perl:native,
+               perl-xs-dev,
+               perl:native

and a patch which does s/pkg-config/ppkg-config/ in Makefile.PL, and
I get:

libnet-dbus-perl_1.1.0-7_i386.deb
---------------------------------
 new Debian package, version 2.0.
 size 183472 bytes: control archive=3128 bytes.
     917 bytes,    19 lines      control
    8013 bytes,    88 lines      md5sums
 Package: libnet-dbus-perl
 Version: 1.1.0-7
 Architecture: i386


I've pushed these changes to a new "cross-945057" branch in
https://salsa.debian.org/perl-team/modules/packages/libnet-dbus-perl.git



Cheers,
gregor

[0] or `use PkgConfig …;' programmatically

Hi gregor,

Thank you for looking into it.

Yes.

That's also what I found.

Good.
look for the construction of @DEFAULT_SEARCH_PATH, you'll notice that it
checks for the existence of dpkg-architecture and effectively inserts
/usr/lib/`dpkg-architecture -qDEB_HOST_MULTIARCH`/pkgconfig into the
path. While that is what we want here, it does rely on environment
variables again. To see this try:

| $ dpkg-architecture -am68k -c ppkg-config --debug doesnotexist
| dpkg-architecture: warning: specified GNU system type m68k-linux-gnu does not match CC system type x86_64-linux-gnu, try setting a correct CC environment variable
| Can't find doesntexist.pc in any of /usr/local/lib/m68k-linux-gnu/pkgconfig /usr/local/lib/pkgconfig /usr/local/share/pkgconfig /usr/lib/m68k-linux-gnu/pkgconfig /usr/lib/pkgconfig /usr/share/pkgconfig
| use the PKG_CONFIG_PATH environment variable, or
| specify extra search paths via 'search_paths'
| $

So we still don't have our single source of truth here.

The other aspect is that this is very much Debian-specific. Given the
effort it takes to make stuff cross buildable, we want to (and do) share
that work with yocto, ptxdist, buildroot and others as much as possible.
However this transformation doesn't help any other distribution.

Last but not least, this approach breaks an important corner case. It
happens every now and then that we need to build a build tool during a
package build. Such build tools need to be built for the build
architecture (to be runnable) rather than the host architecture, so it
is the task of the build system to explicitly tell which pkgconfig files
are requested for the build and which are requested for the host.

Let me give a positive example of how this could work: meson.  When you
want to use pkg-config in meson, you say "dependency('foo')". And when
you want the build architecture pkg-config, you say "dependency('foo',
native: true)". meson then figures how to call pkg-config. Why am I
telling you this? I think that Makefile.PL should also have some syntax
to tell these apart.

Congrats. This sounds like our workflow now mostly "just works"(tm). Let
me know if you run into problems or annoyances.

Do note that cross building from amd64 to i386 isn't that enlightening,
because i386 is runnable on amd64. It's nice to be able do so, but it'll
hide a number of possible problems.

Unfortunately, choosing another architecture is a little difficult at
present:
 * All the mipsen are broken, because our gcc maintainer removed mipsen
   support from the toolchain packages and requested that they live in
   special -mipsen packages. Unfortunately crossbuild-essential-* is
   still missing for all mipsen.
 * Most architectures but armel are currently skewed against amd64 due
   to linux ftbfsing on most architectures.

So at the time of this writing, the only reasonable architecture to
cross build for is armel. I hope we get better weather soon.

This would be better, but what we really need to do here is have
PkgConfig.pm derive its default search path from perl's Config.pm.

Helmut

IMO you're expecting too much from this stuff.

I don't think the upstream Perl community even knows about these cross
build experiments. I'm not aware of any other distros working in this
area either (though I'm happy to be proved wrong).  It's all a pile of
Debian specific hacks abusing upstream features designed for other things.

Sure, it would be nice if ExtUtils::MakeMaker differentiated between
build dependencies for build and host architectures. But that would mean
proper upstream design for cross builds and all and I don't have an itch
to push that myself.

As for a single source of truth: Config.pm is already chosen based on the
(Debian specific) environment variables so I'd say that's still the single
source. I suppose I can bake the host multiarch triplet information into
Config.pm (like I did for debian_abi) if that helps, but it'd still be
Debian specific.

I think passing the right pkg-config in as an environment variable like
your patch does is quite adequate. If there are any actual cases where
the build needs the native pkg-config too, I guess that needs to be
handled as a special case.

Hi Niko,
here.

One other relevant distribution is buildroot. I found that buildroot
packages libnet-ssleay-perl for instance:
https://github.com/maximeh/buildroot/tree/master/package/perl-net-ssleay
This seems to use a "perl-package" build class:
https://github.com/maximeh/buildroot/blob/master/package/pkg-perl.mk
 -> We can see that all build tools are passed as commandline @ARGV
    assignments.

Another relevant player would be PtxDist, but they don't seem packaging
perl modules.

Yocto has a whole meta layer for cpan:
https://github.com/meta-cpan/meta-cpan
The build class here seems to be called "cpan":
https://github.com/openembedded/openembedded-core/blob/master/meta/classes/cpan.bbclass
 -> I don't quite understand how they do cross compilation, but they
    seem to be exporting host architecture dependent PERL_INC, PERL_LIB
    and other things.

For OpenWRT there seems to be a package class "perlmod":
https://github.com/openwrt/packages/blob/master/lang/perl/perlmod.mk
 -> Like with buildroot, build tools are passed on the commandline.

If we count Debian in, this is four different Linux distributions all
trying to cross build (part of) cpan. I think this shows that moving
some of the integration upstream is worth a try. The less each and every
distribution diverges here, the less work there will be.

I understand that you don't want to drive this. A the moment, I'm trying
to figure out what the ideal solution would be. The question of who
drives that solution is secondary to me.

Let me disagree here. dh_auto_configure selects the right Config.pm
using environment variables. That's using an interface between the
builder and debhelper. It's Debian-specific and that's fine, because the
packaging is also Debian-specific. As soon as we call into the upstream
build system (e.g. Makefile.PL), these variable should cease to be a
relevant interface, because this interface should be
distribution-agnostic. So imo, choosing the right Config.pm based on the
environment is fine and still constitutes a single source of truth.

I don't think doing a Debian-specific addition to Config.pm for cross
building is a good solution, because it'd require patching at all ends
and maintaining those patches in Debian.  Either a solution can be
shared with others or we should keep it as simple as possible.

If we assume that our solution cannot be upstreamed, I agree with that.
That'd make me sad though. In a number of (non-perl) occasions, I've
encountered that one of our other cross distributions had fixed a cross
build bug with a patch that wasn't upstreamable (often called
"hackfix"). I've tried sending real and upstreamable patches in such
cases to permanently get rid of the need to patch.

So I think the key here is to propose a useful interface for
communicating pkg-config and then agreeing with all other users on that
interface in order to be able to upstream the resulting per-module
patches.

I guess the next step is searching through our lib*-perl build failures
for occasions of using pkg-config. Then match those failures with other
distributions to encounter prior art and finding a common denominator. I
plan on looking into this, but not today.

Helmut

Thanks for the links, they are interesting. Sorry about my pessimism.

This is of course very commendable.

Thanks for your work.