fabulous cannot be cross built from source as is. The initial problem is
failure to satisfy cross Build-Depends. The immediate offender (sphinx)
is well known and we know that it cannot be marked Multi-Arch: foreign,
because it is sometimes used in architecture-dependent ways. As a
result, client packages must make the decision of which architecture
they need it for. So we annotate sphinx dependencies :native. While that
gets it a bit further, now we see an architecture conflict on python3.
Same problem, same solution: We annotate the interpreter with :native.
This finally lets us build except that xtermspeedup.so is built with the
wrong compiler. dpkg's buildtools.mk can fix that. And finally,
something inside dh-python explodes when the hosts' sysconfig data is
missing due to missing libpython3-all-dev. So here we go with a patch
that fixes everything.

A couple of final remarks. I opted dropping python3-minimal, because it
is implied in python3-all. I also think that you don't actually need
python3-all and that python3 should be sufficient. Even though fabulous
builds a shared object, this object is not dependent on the Python
version, so we only need it once for all versions. Moving fro
python3-all to python3 should remove the need to rebuild fabulous for
Python transitions. I leave this up to you.

Helmut