Dear Maintainer,

I just got bitten by surprising (to me) behavior by the pyproject
plugin: An upstream update to the nmap package has made introducing the
plugin necessary. The nmap package contains two subprojects that have
been built like this for quite some time:

,----
| override_dh_auto_configure-indep:
| 	dh_auto_configure --sourcedir=ndiff --buildsystem=pybuild
| 	dh_auto_configure --sourcedir=zenmap --buildsystem=pybuild
|       # […]
`----

The unpleasant surprise was that the second subproject was not installed
and all I got was this message (and an error from dh_missing late in the
build):

,----
| I: pybuild plugin_pyproject:144: Unpacking wheel built for python3.12 with "installer" module
| W: pybuild plugin_pyproject:150: Scripts directory already exists, skipping unpack. Is the Python package being built twice?
`----

Here's the code responsible:

,----[ /usr/share/dh-python/dhpython/build/plugin_pyproject.py ]
| def unpack_wheel(self, context, args):
|     """ unpack the wheel into pybuild's normal  """
|     log.info('Unpacking wheel built for %s with "installer" module',
|              args['interpreter'])
|     extras = {}
|     for extra in ('scripts', 'data'):
|         path = Path(args["home_dir"]) / extra
|         if osp.exists(path):
|             log.warning('%s directory already exists, skipping unpack. '
|                         'Is the Python package being built twice?',
|                         extra.title())
|             return
|         extras[extra] = path
`----

I'd expect this to EITHER be a warning, i.e. not skip unpacking OR a
fatal error that signals to the surrounding build system that something
seriously needs fixing. (My preference would be the former, but then
again, I'm certain that I don't see the whole picture.)

Cheers,
-Hilko

Hi Hilko (2024.11.29_13:10:43_+0000)

Have you tried with --name? That's the intended way to build multiple
packages at the same time with pybuild.

There's a (somewhat mangled for testing purposes) example in:
https://salsa.debian.org/python-team/tools/dh-python/-/blob/master/tests/tpb07/debian/rules?ref_type=heads

Stefano

* Stefano Rivera:

Thanks for the hint.

I tried replacing every instance of

with

Unfortunately, this does not work in my case. pybuild installs qto
packages python3-ndiff and python3-zenmap, respectively – with no
obvious way to override the package names.

In any case, the warning I mentioned in the bug report ought to be
either not come with any change in behavior or be a fatal error.

Cheers,
-Hilko

Hi Hilko (2024.11.29_22:43:04_+0000)
of this works again. I'll push a commit to improve pybuild.1.

This works: https://salsa.debian.org/pkg-security-team/nmap/-/merge_requests/8

What's going on is that dh_auto_install always specifies a destdir to
buildsystems. So, pybuild's --name mechanism overrides that if it sees
--dest-dir=debian/tmp. If it's set to any other value, it isn't
overriden.

I'll push a git comm to make pybuild.pm cleverer, so it understands if
--dest-dir is specified and doesn't stomp on it with --dest-dir=debian/tmp.

This is harder. It happens a lot, perfectly benignly, in package builds.
That's why it isn't currently a hard error.

To turn it into a fatal error, we'd need to detect that the content in
the scripts directory is different to what we're unpacking. The problem
is that we aren't doing the unpack, we're relying on another module to
do it, so we can't customize the unpack process. We'd probably need to
move the existing directories out of the way, and compare them
post-unpack.

Building multiple python libraries in one source package is *very* rare.
Do you think that's worth the effort?

Stefano