#845498 fp-compiler-#: please improve multiarch and cross-building situation

Package:
fp-compiler
Source:
fpc
Description:
Free Pascal - compiler dependency package
Submitter:
Ben Longbons
Date:
2021-02-21 13:57:03 UTC
Severity:
wishlist
Tags:
#845498#5
Date:
2016-11-23 23:33:00 UTC
From:
To:
Dear Maintainer,

According to `fpc -help`,
-P<x>      Set target CPU (arm,avr,i386,jvm,m68k,mips,mipsel,powerpc,powerpc64,sparc,x86_64)

However, if I try any of those besides the current CPU, I get:

Error: ppc386 can't be executed, error message: Failed to execute "ppc386", error code: 127

(note: while, multiarch would work for x86_64 -> i386, it won't work for
anything else, so you really do need a native ppc386 binary, etc)

You'll probably have to also tell it to use ${triple}-ld, etc.

#845498#10
Date:
2016-11-28 08:19:12 UTC
From:
To:
Dear Ben,

Thanks for reporting this bug. We are aware of this limitation and we have a
plan for fixing this, but unfortunately we lack man power to put this in place.

One of my own goals is to make it possible to compile Android applications as
easy as installing a virtual package that polls everything, which is much more
ambitious than compiling for other architectures using multi-arch.

For now you can use multi-arch to install fp-compiler and RTL packages and then
just use a shell script to call call the right FPC using qemu while calling the
script as expected by fpc binary. Of course this a workaround and native cross
compiler would be preferable, but it should work until someone dos package the
cross compilers.
-- 
Cheers,
Abou Al Montacir

#845498#15
Date:
2016-11-29 01:25:14 UTC
From:
To:
fp-compiler:i386 depends on binutils:i386 rather than
binutils-i586-linux-gnu, and binutils:i386 isn't multiarch
installable. You'd have to do a full cross chroot, currently.

But once the dependency part is fixed, /etc/fpc.cfg can `#INCLUDE
/etc/fpc/$FPCTARGET.cfg` and put `-e/usr/i586-linux-gnu/bin
-Fl/usr/lib/i386-linux-gnu -Fl/lib/i386-linux-gnu -Fl /usr/lib32
-Fl/lib32` in there (each of those .cfg files will have to be manually
written/installed based on the Debian arch (of the fp-compiler
package), the Debian triple (subdir of /usr/lib), the legacy libdir
(/lib32 - actually not sure if this is necessary or not anymore), the
GNU triple (of binutils), and the FPC target (for choice of the
filename)). Incidentally, managing /etc/fpc.cfg through
update-alternatives is a waste since it could be implemented as just
`#INCLUDE /etc/fpc-$fpcversion.cfg` (though since jessie has 2.6.4, an
appropriate upgrade path would need to be written).

The above is fairly trivial and will get you a multiarch "cross"
compiler, with /etc/ ready for real (non-multiarch fp-compiler (I
*think* the libc6-*-cross packages are only needed because of ld.so
conflicting. but fp-units-* are actually multiarch safe already,
they're just not marked as such - they put all their files in
/usr/lib/fpc/$fpcversion/{units,fpmkinst}/$FPCTARGET/ already)) ones.
Then it's just a SMOC to actually build real cross-compilers binary
packages from the Debian source package.

I should probably write a tool to hack-up what I've described above by
using `apt-get download` and extracting/modifying the .deb files.
Maybe test it on Jessie since it has backports to test multiple
*versions* too.

The shell script is unnecessary if you install qemu-user-binfmt (or
qemu-user-static, which `Provides:` it).

#845498#20
Date:
2016-11-29 19:55:29 UTC
From:
To:
Hi Ben,

Wow, I'm impressed with your analysis, you really hit many points that are real
problem for going multiarch.

The dependency on linker package could be fixed easily as you said.

For the /etc/fpc.cfg, this could be solved by adding liens like:
# path to the gcclib#ifdef cpui386-Fl/usr/lib/gcc/x86_64-linux-
gnu/6/32#endif#ifdef cpux86_64-Fl/usr/lib/gcc/x86_64-linux-gnu/6#endif
The file itself is installed by an arch independent package polled by all fp-
compiler packages. This way we have a platform config file

Instead of writing a tool to hack all this, I'd propose you submit patches and
join the maintain team.
-- 
Cheers,
Abou Al Montacir

#845498#25
Date:
2016-11-30 21:29:07 UTC
From:
To:
Hi,
Depends:  binutils | binutils-aarch64-linux-gnu |
binutils-alpha-linux-gnu | .... etc?

Is this instead of what we now add in fp-compiler.postinst? (By the way,
shouldn't we be using ucf for that there?)

You mean arch-dependent, right? fp-compiler-3.0.0 is now creating and
installing that. Should we revise this then?

@Ben, do you think the multi-arch hinter on the tracker is correct
(action needed section of: https://tracker.debian.org/pkg/fpc)

Paul
PS: I'll upload soon for the other bug and stuff that is already
pending. Would be nice if we could get further on this bug as well.

#845498#30
Date:
2016-12-01 20:38:35 UTC
From:
To:
Hi,
available in stretch, it seems most binutils-* packages are now build by
binutils, except the i586 and x86-64 variants, which are build by
cross-binutils. So before we go that route, we should probably first as
for builds of those packages. I was thinking of something like:

Depends: binutils | binutils-aarch64-linux-gnu [ppc64] |
binutils-i586-linux-gnu [i386] |

Would that work to fix this issue?

Paul

#845498#35
Date:
2016-12-02 09:19:28 UTC
From:
To:
Okay, I've got it *almost* working:
https://gist.github.com/o11c/cf98115ba716ebdd1dc2cc75b290f321

I'm still getting errors from update-alternatives in postinst, but I
*think* everything else is right - at least, things that weren't
completely wrong before (there are a lot of those).

I have SUCCESSFULLY produced 32-bit binaries on a 64-bit host, but
that doesn't prove the /etc/ stuff is right. Need to add testcases,
and also force linking with libc.

Hopefully this answers the various questions people have had, e.g.
about what we need to do about binutils.

There are a LOT of TODOs, referring to various bugs (or things) in
various packages (or things), in the script. I suggest you guys look
at them.

Oh, and I haven't tested phase3 with anything besides --dist=stretch.

I will be updating the gist, so remember to `git pull` before.

I'd rather pull my own teeth out with a rusty spoon. Debian packaging
is beyond all doubts the worst I've ever played with - nearly entirely
due to the "in-source" rather than "out-of-source" nature of the
packaging (which is also extremely hostile to upstreams).

Compare e.g. The Fedora documentation [1] - it is *far* simpler for
newcomers, and this is *not* a matter of documentation. (I used to
mention Gentoo since that was my first, but that always makes people
get distracted by the totally-irrelevant fact that the Gentoo project
doesn't *ship* the binary packages). For that matter, even NixOS
packages[2], which live in an utterly alien environment, are easier to
get into than Debian packages.

[1]: https://docs.fedoraproject.org/en-US/Fedora_Draft_Documentation/0.1/html/Packagers_Guide/sect-Packagers_Guide-Creating_a_Basic_Spec_File.html
[2]: https://nixos.org/nixpkgs/manual/

#845498#40
Date:
2016-12-03 02:39:28 UTC
From:
To:
I got it completely working now! I did have to repack
binutils-{i586,x86-64}-linux-gnu though.

Tested that I can generate both i386 and aarch64 binaries, solely by
specifying `-P`. Still haven't actually tested linking with libc, for
that we'll need to do something nasty about /lib32/ (probably just
some ifdefs based on the *host* platform - need to extend the arch
tables for that)

For packaging, you'll almost certainly want to split up the
fp-compiler bits into 2-5 packages
(fp-compiler-{bin,driver,config,config-$fpc_target}). Note that the
new /etc/fpc/debian.cfg must be installed from the *unversioned*
package - which will require a "backwards" dependency
(fp-compiler-config-3.0.0 depends on fp-compiler-config-common).

Updated the GIST at the same URL:

https://gist.github.com/o11c/cf98115ba716ebdd1dc2cc75b290f321

#845498#45
Date:
2016-12-09 21:46:05 UTC
From:
To:
Hi Ben

I am trying to understand you shell script but it contains lots and lots
of overhead which doesn't make reading it easy (the interesting stuff
only starts at line 725). Just to make sure I am not completely
misunderstanding, you are trying to use binutils-<gnu-triplet> package
on the arch of that triplet, right? But e.g. binutils-aarch64-linux-gnu
doesn't exist on amd64. So am I not reading the script correctly or how
is that supposed to work?

Can you explain where this requirement comes from? If really required,
then we'll have to figure out an other solution, because circular
dependencies are a problem.

Paul

#845498#50
Date:
2016-12-10 04:40:05 UTC
From:
To:
You may find it easier to just run it and inspect the resulting
`.deb`s, then refer to the script only when you want to see where a
specific path/package is handled.
https://packages.debian.org/search?keywords=binutils-aarch64-linux-gnu

The *only* packages that are "missing" (actually just outdated and
with wrong headers) are the two x86 arches that are still from
src:cross-binutils rather than src:binutils, for which I filed bug
#846628.
fundamentally wrong, because it is the file read by *all* installed
versions of fpc. In order for each FPC to use its *own* fpc.cfg, they
all need to be conditionally included from a *single* fpc.cfg. Since
jessie shipped with packages that manage /etc/fpc.cfg via
update-alternatives, the symlink must still be managed by it for the
stretch upgrade (for the stretch -> buster upgrade this will no longer
be the case).

Fix: Regardless of version, make /etc/fpc.cfg a symlink to
/etc/fpc/debian.cfg file, which then includes the files specific to
the arch and version.

So the hard dependencies will be:

fp-compiler:$a -> fp-compiler-$v:$a
fp-compiler-$v:$a -> fp-compiler-config-$v:all,
fp-compiler-driver-$v(:$a, but Multi-Arch:foreign)
fp-compiler-config-$v:all -> fp-compiler-config-common:all

(Additionally, fp-compiler-driver-$v should have a backwards
Recommends: fp-compiler-$v:$a and a Description to match, so that
people finding it via `apt-file` (including `command-not-found`) will
get the right thing.)

There is no circular dependency - only the -common package has a
versioned -> nonversioned dependency. And the contents will be:

fp-compiler:$a : empty
fp-compiler-$v:$a : executable /usr/lib/fpc/$v/ppc$a
fp-compiler-driver-$v:$a : executable /usr/lib/fpc/$v/fpc, symlink
/usr/bin/fpc-$v and sometimes (via update-alternatives) /usr/bin/fpc
fp-compiler-config-$v:all : /etc/fpc/$v/{mkcfg,local}.cfg
fp-compiler-config-common:all : /etc/fpc/{debian,local}.cfg, *all*
/etc/fpc/$a/{target,local}.cfg and (via update-alternatives)
/etc/fpc.cfg

It would be possible to put the /etc/fpc/$a/{target,local}.cfg files
in yet *another* package, but IMO there's no value in it. (They are
unversioned so that can't go in fp-compiler:$a which might not be
installed if just fp-compiler-$v:$a is).

While it would *currently* be possible to make fp-compiler-config-$v
architecture-specific (since multi-arch allows overwriting files as
long as they are identical), that would prove to be a mistake once
"real" cross-compiler packages are added. By avoiding relying on this,
it becomes easier to transition from *:$a to *-$a packages in future.

All `Architecture: all` packages should be `Multi-Arch: foreign`.
All `Architecture: $a` packages should be `Multi-Arch: same` *except*
`fp-compiler-driver{,-$v}`, `fp-utils{,-$v}`, and `fp-ide{,-$v}` which
should be `Multi-Arch: foreign` since they only make sense on the
host. (The future fp-compiler{,-$v}-$a packages will also be
`Multi-Arch: foreign`).

#845498#55
Date:
2016-12-12 09:08:57 UTC
From:
To:
Hi Ben & All,
Sorry but this is not the way we accept contributions. We have a big
responsibility against our users to completely ensure the packages we ship are
as safe. as possible This means that we have the duty to inspect all changes
coming from new contributers. Indeed, a big data damage could result on some bad
change that may appear on some system due to some corner cases. Please don't
understand this as if I'm not saying you have bad intension. I'm just saying
that what Paul is doing is the way any Debian mentors shall proceed according to
the DSC.
...
I agree with you here even if when I implemented that, the include statement was
not present in FPC. That was very long time ago you know!
This is a better solution indeed
Looks good proposal
No sure we need this /etc/fpc/debian.cfg if we remove the alternatives in pre-
installation step, but why not. Maybe better  call it /etc/fpc/fpc.cfg
I don't understand why do you need a new package. The fpc configuration file
does not belong to any package, it is created on the fly upon installation.
Can you please explain more your point?
That does not make too much sense.
I'm not fan of overwriting file.
OK on that
-- 
Cheers,
Abou Al Montacir

#845498#60
Date:
2016-12-12 20:37:38 UTC
From:
To:
Hi Paul and All,
What about adding a new package fpc-cross that adds support for cross compiling
with fpc by pulling all these linkers or maybe add a set of fpc-cross-${arch}.
This way if I want to enable cross compiling for armel on amd64 then I would
install fpc-cross-armel
I think there is a better solution. For now we have this at the end of
/etc/fpc.cfg:
This means that every file there will be included.
If we place the regular config file there and we surround it by #ifdef ${arch}
then we can fix this without having all arch configs in the same file

If this is not clear I explain: we replace fpc.cfg to become a regular file that
contains only #CFGDIR directive. Each package will install its config file while
paying attention to surround all its config with the correct #ifdef. This allows
the config to be extended automatically when one installs a foreign compiler.
Yes, sorry for this lapsus
Thanks for the hard work you are doing.
-- 
Cheers,
Abou Al Montacir
# Third party units should be installe in a, multi-arch compatible location.# Units should be installed in /usr/lib/$fpctarget-gnu/fp-units-2.6.2/$pkg/.# Ech fp-units package should install a configuration file called $pkg.cfg in#CFGDIR /etc/fpc-$fpcversion.cfg.d/$fpctarget 

#845498#75
Date:
2017-11-14 22:21:53 UTC
From:
To:
Hi Paul, and All,

I've just pushed a commit [1] that I hope will improve the situation of MA
support.

In this commit I've moved units from ${LIB_DIR}/units/${FPCTARGET}-${FPCARCH}
to ${LIB_DIR}/units/${FPCTARGET}-${FPCARCH}/'$${PACKAGE_NAME}'

where 
LIB_DIR=/usr/lib/${DEB_PACKAGE_NAME}/${DEB_UPSTREAM_MAIN_VERSION}
FPCTARGET=$(CPU_TARGET)-$(OS_TARGET)
FPCARCH=$(shell dpkg-architecture -qDEB_BUILD_ARCH_ABI)

This makes each architecture installs its units in a separate directory.

I've also added a patch against fpcmkcfg to change /etc/fpc.cfg to look for the
units in the right place depending on the cpu/os tragets and the used ARCH/ABI.

I did not upload this as I did not test it extensively and I fear this is a
quite big change that may require manual upload for many arches is not tested
extensively before it is uploaded. I'd prefer to have it in experimental first
also in order to avoid annoyance to our users.

Please let me know what do you think about it.

[1]: https://anonscm.debian.org/cgit/pkg-pascal/fpc.git/commit/?id=a5bfc6ba0fa81
c9e1d627d9314078cb1ddb3d329

#845498#80
Date:
2017-11-14 23:32:00 UTC
From:
To:
Going to experimental before unstable with aggressive changes certainly makes sense. IIRC experimental buildds only use build-depends from experimental when required to satisfy version constraints, so by changing version constraints one can choose whether to build a new version in experimental with the previous version from experimental or with the known-good version from unstable.

However I am skeptical as to whether such an aggressive change is really needed. IMO given the relatively small scale of this problem it makes more sense to leave most architectures alone and only deviate from upstream where we actually need to do so.

I just ran a quick check and currently the only architectures with a conflict are armel and armhf. It seems likely ppc64el would also have a conflict but IMO we can cross that bridge when we come to it.

#845498#85
Date:
2017-11-15 14:33:11 UTC
From:
To:
Hi Peter and All,
Yes, I think this is the way to go in order to avoid the need of re-uploading
binary packages in cas anything goes wrong.
In fact, the biggest interest of MA for FPC is to build for arm and especially
to cross build for Android. So I fear that this is not a tiny goal. Otherwise we
drop MA support which will be a pity.
This is indeed the mail problem. People are asking for better MA support to be
able to use FPC for their phone development and here arm architectures are
important.
Upstream solves this by using a different base dir. We can for instance replace
/usr/lib/${DEB_PACKAGE_NAME}/${DEB_UPSTREAM_MAIN_VERSION}by/usr/lib/${FPCTARGET}-${FPCARCH}/${DEB_PACKAGE_NAME}/${DEB_UPSTREAM_MAIN_VERSION}This is probably less intrusive, but have 2 levels of ${FPCTARGET}-- 
Cheers,
Abou Al Montacir

#845498#90
Date:
2017-11-15 20:01:20 UTC
From:
To:
Hi Abou,

I have a question regarding the patch, as you change generated *.inc
files, can't you generate those in the rules file instead of in the
patch? My experience is that if upstream isn't going to carry the patch
it is hard to update the patch for files like for that every new
upstream release. Also, I am not sure if we aren't already rebuilding
all generated *.inc files in our current build process. I have stripped
major blocks for our patches in the past, because they were totally
unneeded as the file was (or could be) regenerated.

Paul

PS: I would have appreciated it when you would have committed this on a
separate (experimental) branch.

#845498#95
Date:
2017-11-16 10:40:16 UTC
From:
To:
Hi Paul,
That's a very good question and I've asked myself why not regenerate the .inc
file? The issue is that if we do then we end with a possible source change that
is not carried in a patch and this does not please to dpkg-source (or some other
dpkg-* tools I don't remember well anymore). So either we strip this file from
original tar and force regenerating it or we patch it. I'm open to both and know
how to regenerate this file.
Yes this is obvious, but updating the patch can be as simple as: apply the patch
to real source file and then regenerate inc file and then update the patch (kind
of quilt commit)
We are rebuilding some of them.I think many of them are automatically
regenerated by upstream make files.
Sorry I didn't think about it. we can cherry-pick to experimental and then push
-f to master~, but seems our git config refuse forced update.Do you have a clue
other than git revert? otherwise it is also fine.
-- 
Cheers,
Abou Al Montacir

#845498#100
Date:
2017-12-06 09:26:11 UTC
From:
To:
Hi All,

On Wed, 2017-11-15 at 15:33 +0100, Abou Al Montacir wrote:
...
I've uploaded this WE  to experimental a solution that puts files in
/usr/lib/${DEB_PACKAGE_NAME}/${DEB_UPSTREAM_MAIN_VERSION}/units/${FPCTARGET}-
${DEB_ABI}/${FPC_PACKAGE_NAME}

For example:
/usr/lib/fpc/3.0.4/units/i386-linux-base/fpmkunit

However this location of unit files produces the following Lintian errors:
https://lintian.debian.org/tags/arch-dependent-file-not-in-arch-specific-directo
ry.html

I suppose that I need to change it again to be
/usr/lib/${DEB_TARGET_GNU_TYPE}/${DEB_PACKAGE_NAME}/${DEB_UPSTREAM_MAIN_VERSION}
/units/${FPC_PACKAGE_NAME}
So for example
/usr/lib/i386-linux-gnu/fpc/3.0.4/units/fpmkunit


It is now quite easy to do this after my recent changes.

Is that OK or do we need to do it other way, maybe in a simpler manner?
--
Cheers,
Abou Al Montacir

#845498#105
Date:
2017-12-06 21:13:51 UTC
From:
To:
Hi Abou,

For whatever it is worth, that sounds correct to me.

Paul

#845498#110
Date:
2017-12-11 09:20:58 UTC
From:
To:
Hi Paul,
This is now done, we now have unit packages co-installable. However, There is
still an issue with fp-compiler package. It contains some executables that are
installed in /usr/bin while is is MA same according to this report.

As far as I can see fpc-3.0.4, fpcmkcfg-3.0.4, and fpcres-3.0.4 are tools and
should be moved to fp-utils. In this case I'd make fp-compiler recommend fp-
utils.
Then there is grab_vcsa-3.0.4 which is a tool that is specific to video unit,
and that shall be installed by fp-units-base, but this is not my preference as
this just moves the problem to that package. I would rather move grab_vcsa-3.0.4
to fp-utils, and make the fp-units-base recommend fp-utils (maybe not needed as
it recommends fp-compiler).


What do you think about this proposal?
-- 
Cheers,
Abou Al Montacir

#845498#115
Date:
2017-12-15 08:26:20 UTC
From:
To:
Hi Abou,

I don't see obvious flaws in your proposal so I think it is OK.

But I think you didn't fix this in your upload to unstable, so Lintian
is now complaining loudly about all this, as is the MA hinter:
https://lintian.debian.org/maintainer/pkg-pascal-devel@lists.alioth.debian.org.html#fpc
and
https://tracker.debian.org/pkg/fpc

Just curious, why did you upload to unstable without fixing these issues
first?

Paul

#845498#120
Date:
2017-12-15 10:47:41 UTC
From:
To:
Hi Paul
As I explained in my previous message, this will need to take a decision on
either moving files to other packages or to put them in /usr/bin/<triplet>.
I don't generally like big changes in one upload, so I preferred an intermediate
step that moves only unit files.
Also Lasarus new version is being released and I'd like the the new units path
get used in the new release. However on that one I've missed that you already
uploaded it few days ago!
-- 
Cheers,
Abou Al Montacir

#845498#125
Date:
2017-12-15 10:51:40 UTC
From:
To:
#845498#130
Date:
2017-12-16 09:11:50 UTC
From:
To:
Hi Paul,
This looks like a make file was not generated.
I was not aware of this, but I suppose that the fix is easy, I just need to call
fpcmake for that direcotry.
-- 
Cheers,
Abou Al Montacir

#845498#135
Date:
2021-01-12 21:05:31 UTC
From:
To:
Hi,

I'm coming late to the party and I only understand a fraction of what
you wrote, but the parts I do understand make a lot of sense.

I've read the whole discussion and I'm getting the impression that there
are two distinct issues that are conflated inside this bug.

One one hand, it would be nice to be able to install a foreign
fp-compiler to be able to call an emulated (via qemu) compiler. On the
other hand it would be nice to be able to install a cross compiler. And
then there is a third issue not otherwise mentioned, which is cross
building the compiler itself.

All of them are nice and they interact somewhat with one another, but it
helps to tell these issues apart.

This is the part where I only understand very little. The thing I do
understand is that there is a fpc.cfg that influences how the compiler
behaves and given the "right" fpc.cfg it can mostly act as a cross
compiler.

As far as I understand though, there is a major piece missing in the
Debian package to make this reality. I've run fpc hello.pas under strace
and I observe that it calls out to ppcx64. If I were to cross compile
for armel, it would likely call into ppcarm, but there is no ppcarm
binary in any :amd64 package. So regardles how we configure it, we'll
need more binaries to make this happen. Please tell if you think this is
wrong.

I'm not sure yet how you solve the missing binaries, but I'm all for
having cross compilers in the archive.

The initial point was switching the binutils dependency to
binutils-$triplet. Since binutils 2.30-6, we have binutils-for-host and
that means that we always have a binutils-$triplet:$arch where arch and
triplet match. So if you are ok with ignoring oldstable, you can now
depend on the triplet variant without issues.

However, I think that doing so is technically wrong. When I straced fpc,
I saw that it calls into ld.bfd. binutils-x86-64-linux-gnu does not
contain ld.bfd, so the dependency is too weak for what fpc uses. In
order for a binutils-$triplet to be a correct dependency, we'd have to
somehow tell fpc to always invoke triplet-prefixed tools such as
x86_64-linux-gnu-ld.bfd.

And given the pointer to fpc.cfg, I looked into it and stumbled into the
-XP option. It is conditional to cross compiling, but given Debian's
packaging of binutils, it is also relevant to foreign installation.
We'll need to teach fpc to always use the -XP option even natively. Once
that is done, we can replace the binutils dependency with
"binutils-$triplet" or if you require backwards compatibility with
stretch "binutils-$triplet | binutils". Once doing these two bits,
foreign installation should become possible.

The next part was fpc as a cross compiler. I think that one of the first
pieces to getting there is ensuring that all the compiler backend
binaries (e.g. ppcx86 or ppcarm) are built for all architectures. This
does not happen today. Any ideas on how to get there?

A separate problem (at least for me) is figuring out what the API to
cross building with fpc is. What is a builder expected to do to build
natively or to cross build?

How to build sources for some architecture that can be run (aka build
archtecture)?
 * fpc answer: Run plain fpc.
 * gcc answer: Run plain gcc. In future, this requires a dependency on
   gcc-for-build.
 * clang answer: Run plain clang.

How to build sources for architecture $x with triplet $y?
 * fpc answer: ???
 * gcc answer: Run $y-gcc. This requires a dependency on
   gcc-$y. In future, a dependency on gcc-for-host:$x will also suffice.
 * clang answer: Run clang -target $y.

Can someone fill in the fpc answer? I kinda suspect that it is "fpc
-P$z" for some $z that can be computed from $x or $y and the o11c gist
has such a mapping.

Depending on the answer, the packaging layout may need to change. So
I'll stop here.

Hope this helps

Helmut

#845498#140
Date:
2021-01-21 19:30:10 UTC
From:
To:
Hi Helmut,
Better late than never. I'm surprised by the analysis you made, it is quite deep
and correct.
Maybe even three as you wrote below.
This was the idea. But not only, as the foreign  RTL should be usable by both a
foreign or a native cross compiler.
This will require more work on the packaging and I should admit that the current
bandwidth of the team is so low that I fear it will be very difficult. Of course
if someone, with enough free time, volunteers I'll be very happy to help. But I
really have very little time for FPC currently.
For me this is not very useful for users, but only for bootstrapping a new
target. We can ignore it I think.
Maybe a good start is to split this ticket into several ones, each dealing with
a unique feature.
fpc.cfg is a file that gives the path for the RTL (Run Time Library). On Debian
we added a new feature for the compiler to support a #INCLUDE directive. That
one can be used for architecture specific directories. The paths issue is not
only for RTL, but also for the system libraries and the linker configuration. So
a deep analysis is needed. Not sure I have the time to do it for now. But If
done, I can help pointing to the code where we can hack to fix something either
in compiler or in the build system.
fpc is a pure wrapper that calls an architecture compiler called ppc$TARGET.
ppcx64 is native compiler in x86_64, but allows to compile for all system that
run on this architecture (windows, FreeBSD, ...) as long as related RTL is
available.
If you want to compile for arm, then it will call ppcarm, which is an arm
architecture compiler. It can be a cross compiler, but it can be a shell script
that calls the native foreign ppcarm with quemu. This needs to be further
analyzed for pros and cons. The same ppcarm should also be able to compile for
all arm related OSes like Android ...

However there is still a problem to be solved as we call ppcarm the compiler for
armel and armhf. So there will be a need to hack this in fpc code and send the
patch to upstream.
The missing binaries can be first emulated by qemu with foreign compilers, and
later with a native cross compiler it ever needed.
I don't think old-stable is important anymore. However the real issue, as you
say later, is that we can not use ls, but ld.bfd.
We already patch the compiler to call ld.bfd instead of ld, so we can change the
name as you wish.
I should admit this is new for me, but looks feasible. I can help patching the
compiler code to force using -XP as required.
You need to compile the source package with special flags to generate a given
compiler.
So this will be as if you build several separate compilers.

I would advise to make the cross compilers as separate source package that
relies on FPC source. Otherwise it will be too complicated to package any new
version. But it is feasible.
The answer is ver easy; fpc -P$z where $z is the suffix of the arch
(cross)compiler. The suffixes are defined by upstream, but the rules file
manages to detect them for each dpkg arch using a mapping table. We cal also add
a helper program that returns fpc arch from dpkg arch.
Yes It helped a lot. thanks a lot and don't hesitate to correct me if I
misunderstood some of your comments.

#845498#145
Date:
2021-01-22 09:06:27 UTC
From:
To:
Hi,

Can you implement just this part and poke me once that has hit unstable?
I can send a patch for the next step then.

Both models are used in Debian and I've gained some experience with
both. For instance, binutils uses the "lump everything
together"-approach. And gcc is split into several source packages
(gcc-10, gcc-10-cross, gcc-10-cross-ports). In my experience the cross
packages go out of sync way too often and I strongly prefer the combined
approach. Why do you think that is getting too complicated?

The way I see things is that rather than building fpc once, the Debian
package would build it a number of times. The build time would go up
significantly, which is a disadvantage, but testing time can be reduced
using build profiles. I think I can help contain this approach.

This is a mix of clang and gcc approaches. Like clang, the target is
specified using a command line switch. Like gcc, we need one binary
executable for every combination of build architecture and host
architecture (simplified, it's a bit less). As such, I think we should
provide packages fpc-${DEB_HOST_ARCH} that makes fpc -P$mapped_arch
work. In a first iteration, we'd just do the "diagonal" of the matrix
(i.e. no cross tools, just working with emulated compilers). The rest
can be filled in later on demand. Cross building fpc itself strongly
depends on a filled matrix, so that part comes last.

Sounds like we do have an implementation sketch now. :)

1. Make fpc call into triplet-ld.bfd instead of ld.bfd.
2. Change the binutils dependency to binutils-triplet | binutils.
-> Foreign installation of an emulated fpc should work.
3. Provide suitable fpc-arch packages from fpc.
4. Add fpc-for-host and fpc-for-build metapackages.
5. Add cross compilers.
-> Cross building with fpc should work.
6. Make fpc cross buildable.
-> Cross building fpc should work.

The devil is in the detail of course.

Helmut

#845498#150
Date:
2021-02-21 13:52:08 UTC
From:
To:
Hi Helmut,
I did not really patch this, but while looking for such a way to patch, I
discovered a nice command line option -XP.
This ca be used as follows:
fpc -XPx86_64-linux-gnu- myprogramfpc -Pi386 -XPi386-linux-gnu- myprogram
It should be possible to hack the default value of this CLO so that we can make
it point to the right triplet.
Does this fit your need?