Our pyproject (PEP517) build system now seems to be operating reliably. Using the old default system with setup.py, we now get reminded that it is deprecated. In some cases the old default setup method is even "harmful" in the sense of running configuration twice, once for the build rule and again for the install rule (arguably this could be seen as a bug in the setup.py). We had to introduce pyproject (pybuild-plugin-pyproject) when python modules started appearing with a pyproject.toml and no setup.py file. But now we're in the reverse situation: a pyproject build can proceed (using rules in setup.py) even if there is no pyproject.toml file. So pyproject builds now seem to be the superior option, even when there is an old setup.py file with no pyproject.toml. For that reason I propose switching pybuild to start using pyproject as the default system rather than distutils. (I expect this would need dh-python Depends: pybuild-plugin-pyproject)
I think this proposal is at least one release cycle too soon. We're close to freeze and in the midst of changing the default python version. At the very least this should wait until Bookworm is out. There should also be a rebuild of the affected packages to see what problems are induced. Maybe, after Bookworm is out, we could get the reproducible builds people to do a special run with a setup.py build followed by a pyproject.toml build so we can see the differences. Scott K
Mind you, the build module for PEP517 builds seems not to give good control of build and build-temp dirs. For example, the temp dir for building python extensions gets placed in build rather than .pybuild, which is not in the spirit of the pybuild tool. python3 -m build does not seem to have options for controlling the location of the build dirs. It might be possible to pass it to the underlying setup.py using the -C flag, but I couldn't get that to work easily from debian/rules It would require running pybuild manually with a separate -p option for each python version. Perhaps this is a feature which could be addressed in pybuild. It matters in particular for python modules which are rebuilt with different configurations, for different debian python packages from the same source. For instance petsc4py provides separate real and complex number builds of its python module. So perhaps it's premature to use pyproject by default until there's better control of the build dirs used for pybuild. On the other hand, perhaps it's nevertheless still worth setting pyproject as the default system, to accommodate the simpler packages. The petsc4py build can't really be called "simple", so doesn't necessarily need to influence discussion of the default system. It needs to retain a distutils build for now, which can be managed from its debian/rules.
Hi Drew (2023.01.07_07:56:43_-0400) I think this is something worth a try in early trixie. It'll likely be a change of default mode, and require packages to explicitly the request the setuptools mode. Does that matter? They get built once, each. Both temporary directories don't need to exist at the same time. Look at this example of building two PEP-517 modules from the same source package: https://salsa.debian.org/python-team/tools/dh-python/-/blob/master/tests/tpb07/debian/rules In this case, they are in separate source directories, but the same thing should work for separate builds with different configuration. Each build should end up in its own .pybuild directory, even if temporary directories are shared. SR
On Mon, 30 Jan 2023 12:12:48 -0400 Stefano Rivera <stefanor@debian.org> wrote: > Hi Drew (2023.01.07_07:56:43_-0400) > > I think this is something worth a try in early trixie. > It'll likely be a change of default mode, and require packages to > explicitly the request the setuptools mode. > > > It matters in particular for python modules which are rebuilt with > > different configurations, for different debian python packages from > > the same source. For instance petsc4py provides separate real and complex > > number builds of its python module. > > Does that matter? They get built once, each. Both temporary directories > don't need to exist at the same time. I'm just curious if switching to pyproject will require packagers (that is, users of dh-python) to change anything in their python packages, or if this change will be completely isolated inside of dh-python?
I've started collecting ways in which this can go wrong in https://wiki.debian.org/Python/PybuildPluginPyproject. Note that they definitely include "package continues to build successfully but now ships problematically different contents" (generally easily fixed, but requires manual attention). I'm fairly sure that I've seen "package silently stops running any tests during its build but succeeds anyway" as well, though I don't have an example of that right now. I'll extend this page as I notice other problems. FWIW, I agree that we need to do this one way or another. The intent of this page is to help people fixing packages that break, and perhaps also to guide some kind of effort to analyse the remaining problems in bulk and do a mass bug filing. I'd like to have time for the latter, though I'm not sure whether I will.