Hi, I think this should be in a bug report in the BTS to track it in a better way. I was trying to test `pygrub` and found the Python 3 version is definitely broken in the 4.14 packages. I was able to get the script to display the help message by adding "/usr/lib/xen-4.14/lib/<triplet>" to sys.path. The existing line: sys.path.insert(1, sys.path[0] + '/../lib/python') Is distinctly odd, usually this is better expressed: sys.path.append(os.path.join(sys.path[0], "libexec")) (though I suppose we can assume Linux, but this is Bad Practice) The way some portions of `pygrub` are packaged are distinctly odd. Certainly xc.so and xs.so are linked to core Xen libraries and need to be version-specific. Yet libfsimage.so appears independant of the Xen version and should likely track `pygrub`, rather than matching the system Xen version. ---->8---- Forwarded Message ---->8---- I also have a little snippet from IRC, which is about this, where Ian reports that he's seen it working. https://salsa.debian.org/xen-team/debian-xen/-/snippets/500 So, apparently there are cases in which pygrub 'works' and in which it does not, and apparently using pygrub with "amd64 kernel and Xen tools but i386 userland" is problematic, and I remember some remarks which I can't find back about that that use case was probably already broken always, in the past. I wanted to find out about this and set up some test cases to reproduce things (I've never used pygrub yet), but that obviously did not happen yet. I have some stuff going on in my personal life that is taking up a lot of time currently. What is rather easy for *me* is to help organizing the work and managing todo lists etc, but not learning new stuff ATM. So, my current questions are: 1. Is pygrub a blocker for having Xen 4.14 in unstable? Because that should be our first team-goal now. 2. What exactly is going on, can we make a list/table/whatever about in which cases pygrub 'does not work' (in more detail, how does it fail). 3. pygrub keeps being the thing that always causes problems. What would be your (asking anyone who wants to think along) ideas about which well-defined situations/test-cases we should have to execute instead of having the users report problems after big package changes? Hans P.S. Next message after the commercials will be on #968965 which is the other biggest issue for Xen 4.14 in unstable now.
Due to working with Pry Mar, I can state the cross-compilation of the Python shared objects may not be 100% functional yet. Looks very much like Python's "distutils" took 2 steps forward and then 2 steps backward during the Python 2 -> Python 3 transition. (Great! Linking and compilation got separated. Ewww! CFLAGS gets appended to LDFLAGS. Great! We'll add support for architecture-specific compilation directories. Ewww! There isn't a good way to pass in the architecture triplet.) I've got an initial patch for working around an issue here, but the quality doesn't look great to me. Something along those lines should be submitted to Xen, but I'm unsure of all the issues.
Hans van Kranenburg writes ("Bug#975062: Python 3 (pygrub) in 4.14 packages"):
The problem with pygrub with 32-bit userland is as follows:
* Xen has to be 64-bit since there is no 64-bit Xen.
* dom0 kernel bitness and Xen tools bitness must match because
Xen 32/64 compat ABI understands only one bitness for dom0
and Xen dom0 tools make hypercalls directly so must match
the kernel.
* 32-bit kernels are starting not to be able to drive hardware
(big PCI bars, bugs, etc.) so you want a 32-bit kernel.
* So you must have libxen*:amd64.
* pygrub uses python, obviously. It needs to load the xenfsimage
library, which is part of xen tools, since that is the userland
library tht understands the guest filesystem to fish out the guest
kernel.
* The xenfsimage library is not in its own package [1] - it's in with
some other Xen libraries. But it is going to be loaded into a
python interpreter, so it needs to match the bitness of the python
interpreter.
* You can't install a 64-bit python interpreter without basically
doing the whole 32-to-64 crossgrade. (That crossgrade is what I
ended up doing on my home machine.)
* You can't co-install libxen*:i386 because the Xen libraries aren't
properly multiarched. [2]
* This gets worse now that the Xen packages use python3. Previously
with a minimal but modern system you might be able to get away with
having a 64-bit python2 and a 32-bit python3.
Both [1] and [2] are in principle bugs in the Xen packages. Upstream
sentiment seems to be that 32-bit userland is not really a very good
idea any more anyway. So we could solve this by fixing [1] or [2] or
we could expect people to use 64-bit dom0 userland (and crossgrade if
need be).
I think yes, a working pygrub ought to be a blocker for 4.14 in
unstable.
But I think we have that - I rebuilt the existing packages for buster
and it WFM.
IDK about any other problems than the bitness one above.
Ian.
Due to working with Pry Mar, I can state the cross-compilation of the Python shared objects may not be 100% functional yet. Looks very much like Python's "distutils" took 2 steps forward and then 2 steps backward during the Python 2 -> Python 3 transition. (Great! Linking and compilation got separated. Ewww! CFLAGS gets appended to LDFLAGS. Great! We'll add support for architecture-specific compilation directories. Ewww! There isn't a good way to pass in the architecture triplet.) I've got an initial patch for working around an issue here, but the quality doesn't look great to me. Something along those lines should be submitted to Xen, but I'm unsure of all the issues.
It seems I was distracted when I wrote this mail.
Ian Jackson writes ("Re: Bug#975062: Python 3 (pygrub) in 4.14 packages"):
^^ 32
^^ 64
Hi!
^^ 32?
^^ 64?
My personal opinion is that there are more interesting horses to fry (or
what was the saying) for our very bandwidth limited team. So yes, let's
move this from the perfection into the known issues department for now
(bullseye).
When someone shows up with a real world issue who's really panicing
after recklessly upgrading (after the bullseye release halfway 2021) we
probably might help by giving pointers and instructions. Until that
actually happens, we should not spend time writing documentation about
how to do that etc.
OK.
Ok, thanks a lot for the write up, both, and now we have a debian bug to
look back to which is a bit easier to track than mailing list messages.
So, I'm crossing out this issue now as blocker.
Hans
HAH, thanks. \:D/ Hans