#1081956 cross-autopkgtests fail to import sysconfigdata

#1081956#5
Date:
2024-09-16 16:20:59 UTC
From:
To:
Ubuntu's autopkgtests for i386 run in a cross environment, with many
amd64 packages installed.

pybuild-plugin-autopkgtest exports some environment variables to assist
with cross building (#892931), but these break the cross autopkgtests.

e.g. https://autopkgtest.ubuntu.com/results/autopkgtest-oracular/oracular/i386/n/nxmx/20240914_133446_33f49@/log.gz

All the supporting packages are installed from amd64, not i386, not a single i386 package is mentioned in the autopkgtest log.

But:

 78s ==================================== ERRORS ====================================
 78s _____________________ ERROR collecting tests/test_nxmx.py ______________________
 78s ImportError while importing test module '/tmp/autopkgtest.O1QJkE/autopkgtest_tmp/build/tests/test_nxmx.py'.
 78s Hint: make sure your test modules/packages have valid Python names.
 78s Traceback:
 78s /usr/lib/python3.12/importlib/__init__.py:90: in import_module
 78s     return _bootstrap._gcd_import(name[level:], package, level)
 78s tests/test_nxmx.py:11: in <module>
 78s     import nxmx
 78s /usr/lib/python3/dist-packages/nxmx/__init__.py:15: in <module>
 78s     from scipy.spatial.transform import Rotation
 78s /usr/lib/python3/dist-packages/scipy/__init__.py:67: in <module>
 78s     from scipy._lib import _pep440
 78s /usr/lib/python3/dist-packages/scipy/_lib/__init__.py:12: in <module>
 78s     from scipy._lib._testutils import PytestTester
 78s /usr/lib/python3/dist-packages/scipy/_lib/_testutils.py:22: in <module>
 78s     _v = sysconfig.get_config_var('HOST_GNU_TYPE') or ''
 78s /usr/lib/python3.12/sysconfig.py:782: in get_config_var
 78s     return get_config_vars().get(name)
 78s /usr/lib/python3.12/sysconfig.py:765: in get_config_vars
 78s     _init_config_vars()
 78s /usr/lib/python3.12/sysconfig.py:706: in _init_config_vars
 78s     _init_posix(_CONFIG_VARS)
 78s /usr/lib/python3.12/sysconfig.py:572: in _init_posix
 78s     _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0)
 78s E   ModuleNotFoundError: No module named '_sysconfigdata__i386-linux-gnu'
 78s =========================== short test summary info ============================
 78s ERROR tests/test_nxmx.py
 78s !!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
 78s =============================== 1 error in 0.26s ===============================

Presumably because environment variables were set for cross-compilation,
triggering this code:

https://salsa.debian.org/python-team/tools/dh-python/-/blob/b71d65b7b0b510bb6f90e2d3e98e35229b6c7c50/pybuild#L73-91

My assumption is that entire block should be skipped if we're running with
--autopkgtest. But I am not at all familiar with this cross autopkgtest environment.

Stefano

#1081956#10
Date:
2024-09-16 18:07:05 UTC
From:
To:
Hi Stefano,

For cross compilation matters, please Cc debian-cross@l.d.o instead of
me in future, thanks.

This very much looks like something (probably pybuild-plugin-autopkgtest
as you say) exported _PYTHON_SYSCONFIGDATA_NAME. The exported value
'_sysconfigdata__i386-linux-gnu' is sensible for i386. The failing
import indicates that libpython3.12-dev:i386 has not been installed.

Yes, when the dpkg-architecture variables are exported, dh-python
detects cross building and does its thing.

I'm not convinced, but I know too little about the mechanics. My naive
assumption would be that if we want to cross test a Python module for
i386, we should likely use the i386 Python interpreter. Otherwise, we
are effectively doing a native test rather than a cross test. So the
premise of installing python3.12 looks broken to me or I don't
understand cross testing.

Helmut

#1081956#15
Date:
2024-11-18 18:00:05 UTC
From:
To:
Hi Debian (2024.09.16_18:20:59_+0200)

I forgot about this for a while, but I've come back to the conclusion
that something is horribly wrong higher up in the stack here, not down
in dh-python.

There is no value here in testing a pure python module (python3-nxmx)
inside an amd64 python, on i386, surely.

If that's being tested, it means either the Depends are invalid, or the
way the test is constructed is invalid.

python3:any as a Depends on a pure-Python module seems perfectly correct
to me. So I think either this test shouldn't have been run at all, or
should have been run with python:i386.

autodep8 outputs this on nxmx:
Depends: @, pybuild-plugin-autopkgtest, @builddeps@,

Which seems right. I don't know how cross autopkgtest decides what needs
to be native.

Stefano