#1071601 python3-setuptools: Prefix is not exactly honored when installing with --prefix

#1071601#5
Date:
2024-05-22 04:05:20 UTC
From:
To:
Dear Maintainer,

In a simple distutils project:

$ cat hello
#!/usr/bin/env python3 print("Hello world")

$ cat setup.py
from distutils.core import setup
setup(name='hello', scripts=['hello'])

Installing it with --prefix is not fully honored, as "local" is always
added (--root is optional, used just to make the reproducer easier):

$ python3 setup.py install --root=/tmp/py-root --prefix=/opt/py-prefix

$ find /tmp/py-root/
/tmp/py-root/
/tmp/py-root/opt
/tmp/py-root/opt/py-prefix
/tmp/py-root/opt/py-prefix/local
/tmp/py-root/opt/py-prefix/local/bin
/tmp/py-root/opt/py-prefix/local/bin/hello
/tmp/py-root/opt/py-prefix/local/lib
/tmp/py-root/opt/py-prefix/local/lib/python3.12
/tmp/py-root/opt/py-prefix/local/lib/python3.12/dist-packages
/tmp/py-root/opt/py-prefix/local/lib/python3.12/dist-packages/hello-0.0.0-py3.12.egg-info
/tmp/py-root/opt/py-prefix/local/lib/python3.12/dist-packages/hello-0.0.0-py3.12.egg-info/dependency_links.txt
/tmp/py-root/opt/py-prefix/local/lib/python3.12/dist-packages/hello-0.0.0-py3.12.egg-info/SOURCES.txt
/tmp/py-root/opt/py-prefix/local/lib/python3.12/dist-packages/hello-0.0.0-py3.12.egg-info/top_level.txt
/tmp/py-root/opt/py-prefix/local/lib/python3.12/dist-packages/hello-0.0.0-py3.12.egg-info/PKG-INFO

In fact in such case everything should be installed in
/tmp/py-root/opt/py-prefix prefix and not
/tmp/py-root/opt/py-prefix/local since a prefix is explicitly defined
and so there's no point to add an extra "/local" subpath.
---

In between the others, this breaks jhbuild builds as a test fails for
this specific reason (see bug #1054720 where the missing file is due to
the fact that it gets installed to $PREFIX/local/bin instead of $PREFIX/bin).
--- After some debugging of it, this related to the default scheme change happened with python3.10 [1] which meant to always use /usr/local as default and ensure that one should explicitly require for /usr prefix, but in the case that --prefix= is used, the user request is now ignored because: - /usr/lib/python3.*/_distutils_system_mod.py: 1. Selects the scheme `posix_prefix` since we've requested a prefix explicitly - /usr/lib/python3/dist-packages/setuptools/_distutils/command/install.py 1. Calls sysconfig.get_preferred_scheme("prefix") that now as per the said change would return by default `posix_local` unless we're building a debian package. 2. Replaces the variables to use the `posix_local` scheme So, in order to fix this we may in theory change python3 packages to make 1. to select another defined prefix say `posix_explicitprefix`, and then making `sysconfig.get_preferred_scheme("explicitprefix")` to instead return `posix_prefix`, but that would not be correct, because being that a public API it should only support the allowed input arguments for it ("prefix", "home" or "user"). As per this, I feel the best way to handle this is instead in the setuptools module so that we can check once again if --prefix has been used and react accordingly. [1] https://lists.debian.org/debian-python/2022/03/msg00039.html [2] https://bugs.launchpad.net/ubuntu/+source/python3.10/+bug/1967920
#1071601#12
Date:
2024-05-22 04:29:16 UTC
From:
To:
Adding a patch that fixes this issue by using the `posix_prefix` schema
when the install prefix is explicitly requested

#1071601#19
Date:
2024-05-22 05:09:56 UTC
From:
To:
Oh, actually an even more conservative approach can be by also honoring
the DEB_PYTHON_INSTALL_LAYOUT env variable, because if that is set it
should have the priority so that `lib/python3/dist-packages` is used
instead of `lib/python3.XX/site-packages`