My upstream project (cockpit) is soon going to deliver a new Python component,
where it installs a locally built wheel (i.e. some Python packages and
auto-generated executable wrappers) with

    python3 -m pip install --no-index --force-reinstall --root='$(DESTDIR)/' --prefix='$(prefix)' tmp/pybuild/dist/*.whl

This works fine on Fedora, but not on Debian:

   rm -rf /tmp/i
   python3 -m pip install --no-index --force-reinstall --root=/tmp/i --prefix=/usr tmp/pybuild/dist/*.whl
   find /tmp/i

This shows that everything gets installed into /tmp/i/usr/local , i.e. /bin,
lib/python3.11, etc.

I'm quite sure this is somehow related to the Debian magic of not installing
into /usr [1][2], but this is happening during *building* a Debian package,
with --root.

It seems to me that (1) explicitly specifying --prefix=/usr should still be
respected, and not silently be changed to /usr/local (as a *default*,
/usr/local/ is fine of course). And (2) perhaps this change shouldn't be
applied with --root?

Do you know a workaround for this? I can't simply move debian/tmp/usr/local/*
to debian/tmp/usr/, as the upstream build system already assumes that a wheel
binary lands in DESTDIR/usr/bin/, i.e. `make install` is just broken.

Thanks,

Martin

[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=771794
[2] https://salsa.debian.org/python-team/packages/python-pip/-/blob/master/debian/patches/hands-off-system-packages.patch

Hello all,

Is that patch even necessary still? In Fedora with python3-pip 2.22.3,
`sudo pip install python-dbusmock` installs into
/usr/local/lib/python3.11/site-packages/dbusmock/ , i.e. does not conflict
with /usr any more. They do have a more targetted protection of /usr [1], but
no general path mangling in their other patches [2].

Thanks,

Martin

[1] https://src.fedoraproject.org/rpms/python-pip/blob/rawhide/f/remove-existing-dist-only-if-path-conflicts.patch
[2] https://src.fedoraproject.org/rpms/python-pip/tree/rawhide

Hello again,

I just locally reverted [1], and it's still the same, so that Debian patch
actually isn't to blame.

I also tried --prefix=/opt which then gets me /opt/local/{bin,lib}. So this
behaviour directly contradicts the --help documentation:

It's also not about my local wheel --

    rm -rf /tmp/i; python3 -m pip install --root=/tmp/i --prefix=/opt small-timer
    find /tmp/i

again gives /tmp/i/opt/local/lib/python3.11/dist-packages/small_timer

So this "/local" thing is both sticky and hard to avoid..

Martin

[1] https://salsa.debian.org/python-team/packages/python-pip/-/blob/master/debian/patches/hands-off-system-packages.patch

Hi Martin (2023.05.05_13:03:43_+0200)
...

Yeah, the cause of this is our posix_local sysconfig scheme.

Ideally pip should have a --scheme argument. At least in Debian, where
we have multiple schemes interacting with each other, that could make
sense.

Or pip could use some knowledge of Debian's schemes to select
posix_prefix / posix_local as appropriate...

This all gets messy, I'm afraid.

Stefano

A note on the effects of this, I'm managing a largeish
commercial scientific software package designed to run
on Debian and derivatives, it has a bit of Python in
it.  We install the package, then the users have full
range on the machine.  To fix deprecation warnings
about

   python setup.py install

I tried the documented replacement

   pip3 install . --prefix=/usr/local

and was baffled to find no files in /usr/local/bin etc,
a "find / -name ..." found them in /usr/local/local as
per discussion above.  To work around this, I guess I
could use --prefix=/usr, but that seems particularly
dangerous, all it needs is someone on the machine to run
pip3 --update pip and that above command installs into
/usr.  Some kind of autoconf detection of whether the
pip3 we've got is "debian-extra-local"-ised would seem
the only option, gnarly.

This becomes pressing as the deprecation of setup.py
installations progresses of course.

Best wishes

Jim

tags 1035546 + patch

I submitted https://salsa.debian.org/python-team/packages/python-pip/-/merge_requests/1
as the patch to address this issue.

ciel.